decrypt.co
Amid a rising market for Solana NFT collections—headlined by recently hot projects like Okay Bears and DeGods—has come an increase in malicious programs used to unfairly game new NFT launches on the blockchain. On April 30, it crashed the entire Solana network.
On that Saturday, such programs (or bots) sent an automated barrage of transactions—6 million per second, according to a postmortem report by Solana Labs—that overwhelmed the blockchain network. They did so in an attempt to beat out legitimate users during the minting process, in which collectors purchase newly generated NFT collectibles from a project.
The bots swarmed Candy Machine, the minting tool for Solana NFT protocol Metaplex, and took down the whole network in the process. Solana was completely unusable, causing a mad scramble amongst validators and contributors to diagnose the issue and bring the network back online. It took seven hours to restore service to Solana.
The rise of NFTs—unique tokens that are used to demonstrate ownership over digital assets like collectibles—has driven significant interest to platforms like Solana and Ethereum over the past year. While Ethereum has the largest such ecosystem, yielding billions of dollars in NFT trading volume each month, Solana is rapidly gaining traction.
In April, Solana NFTs collectively yielded $295 million in sales, per data from DappRadar, marking a 91% month-over-month jump. All told, according to CryptoSlam, Solana has seen over $2.2 billion worth of NFT trading volume to date, and the rising market appears to be driving significant user and developer activity to the platform.
Part of the reason Solana has gained traction in the NFT space since last fall is due to its speedy transactions and extremely low fees—real differentiators from Ethereum. However, that makes the network “kind of amenable to bots,” Metaplex Studios CTO Nhan Phan told Decrypt.
In other words, some of the key aspects that currently make Solana appealing for NFT buyers also make the platform susceptible to attack. It’s not the first time that Solana’s network has collapsed under enormous strain—last September’s downtime, blamed on a token launch at a DeFi protocol, lasted over 17 hours. But it’s the first pinned on NFT-related activity.
After the crash, Metaplex and Solana Labs both began sharing plans to address the NFT botting situation and avoid this issue in the future. And it starts with a “bot tax.”
Make the bots pay
Rolled out soon after Solana came back online, Metaplex’s botting penalty charges such programs a fee (or tax) for submitting “invalid” transactions—that is, masses of failed transactions that are identified as coming from an automated program that is “blindly trying to mint,” per tweets from the company.
Prior to its implementation, there was no real downside to people using botting programs to flood Candy Machine mints to buy new NFTs and box out other, legitimate users. If a bot sent out 100,000 transactions during an NFT mint, for example, and successfully minted 100 NFTs from completed transactions, there would be no penalty for the other 99,900.
“They were trying to hammer the Candy Machine program just as the mint was opening, so none of the users could get in,” said Phan. “And then they would just leave their bots on forever, which is annoying for everyone. Because there was no real monetary impact to them, they were just like, ‘OK, whatever, it doesn't matter.’”
Today #Solana mainnet-beta went down partially due to botting on the Metaplex Candy Machine program. To combat this, we have merged and will soon deploy a botting penalty to the program as part of a broader effort to stabilize the network. https://t.co/QaAZT3VxXz
— Metaplex (@metaplex) May 1, 2022
Now there’s a 0.01 SOL tax attributed to such transactions that are identified as being from bots, and while that’s a small individual fee—about $0.50 today, amid a tumultuous crypto market—it could really add up for anyone attempting to overwhelm NFT mints on Solana.
As of Wednesday, the tax had generated 1,620 SOL (nearly $82,000) in penalties, and the funds go to the creators of each respective NFT mint affected by bots. “Finally, creators are being compensated by the bots, which is a pretty interesting and funny thing,” said Phan.
On top of that, the Metaplex protocol changes help make the Solana network more efficient in another way. Phan said that the update enables validators to rapidly process and validate such transactions, “rather than go through the churn of trying to agree on whether it's an invalid transaction.” In other words, it reduces the load of botting on the wider network.
According to Phan, malicious botting in NFT mints on Solana has “gone down by an order of magnitude” since the Metaplex update. However, the SOL tally above suggests that some botters are still trying to game the system with about 162,000 detected NFT botting transactions.
Conflict amid chaos
Amid the frenzied rush to bring Solana back online April 30 and May 1, network validators were supplied restart instructions that included the option—but not requirement—to temporarily block the Candy Machine minting program to avoid further botting issues during the restart.
Additionally, there was chatter among validators of blocking other NFT-related smart contracts—the code that powers NFTs and decentralized applications—at the time, including that of leading Solana marketplace, Magic Eden. The marketplace’s founder, Jack Lu, wrote in a Discord post to validators, “Pls DO NOT BLOCK THIS ADDRESS PLS.”
Magic Eden founder begging Solana validators not to block their contracts.
this is bad. pic.twitter.com/f4d5UKACIE
— Can Gurel (@CannnGurel) April 30, 2022
"On Saturday [April 30], we were notified that validators were discussing shutting down our contract while keeping other marketplaces' contracts open,” Lu told Decrypt in a statement. “To us, this was not espousing the values of Web3.”
“We also found that there was unclear methodology by which these decisions were being made, so we had no choice but to jump in in real time and try to protect our marketplace from being shut down selectively and at the exclusion of other marketplaces,” he continued. “Luckily our marketplace contract was not blocked and the network has since been restored."
Even so, the idea that Solana validators were asked to consider blocking NFT-related apps and services received significant pushback across social media, with tweets decrying perceived censorship on the Solana network. While intended as temporary, the idea of censoring transactions at the layer-1 level could be seen as a slippery slope.
The instructions were apparently written and compiled by validators in the community, but were shared widely by the Solana Foundation and founders of the network.
Hudson Jameson, formerly of The Ethereum Foundation, tweeted that such parties sharing the instructions constituted “an endorsement of optional compiled instructions to censor.” He added that it "sets precedents around censorship at L1."
Oh fuck. Solana Foundation published beta cluster restart instructions that include instructions to block NFT minting bots manually at the L1 layer.
🫣
"Don't forget, only censor the network until it looks steady again so we can start saving face with an online network." https://t.co/LDpSuunbzt pic.twitter.com/BF6wuZDE9I
— Hudson Jameson (@hudsonjameson) May 1, 2022
Austin Federa, head of communications at Solana Labs, repudiated many such tweets personally, in one case tweeting that Lu’s screenshotted plea was “taken completely out of context.” Federa added, “One of the great things about an open permissionless community is anyone can propose anything for any reason.”
Phan described the validators’ instructions as a “temporary, optional blocking” designed to help get the network up and running again.
“It was absolutely our intent to unblock everything,” he added, noting that the instructions were really focused on Candy Machine. “This was mainly a collaboration between us and the validators in the interest of the ecosystem.”
It’s an arms race
With the “bot tax” live and the Solana network not experiencing significant issues since the network returned online on May 1, it looks as though the NFT mint botting issue has been addressed—for now, at least.
However, Metaplex and Phan aren’t content to assume it will stay that way forever. He described the tit-for-tat battle between botters and protocol developers as “an everlasting arms race,” and expects that people seeking advantages in minting potentially valuable NFTs will continue to work to find ways around the latest changes.
“As long as there's economic incentives [...] to go and do botting, then there will be botting, right?” he said. “I think the beautiful thing about decentralized, censorship-resistant systems is that anyone can kind of do this stuff. That's also one of the cons.”
He described the botting penalty as “step one, of many steps” to protect NFT mints and the wider Solana network as a result, but wouldn’t detail potential next moves: “If I shared them, then the bots would know.”
Ongoing relief may ultimately come from Solana reworking its fee model to let users effectively bid for priority when submitting transactions. Solana’s postmortem on the April 30 crash asserted that “fees are coming to Solana,” but it’s really “fee prioritization” that’s coming. Spamming the network could prove to be incredibly expensive once changes are implemented.
In the meantime, Metaplex isn’t letting the Candy Machine-led network crash keep it down. The team is working on a next-generation NFT specification, which will include performance improvements, cost reductions, and modularity features.
Phan said that the new specification will enable creators to pursue “all of the awesome, amazing, and crazy new things that people are trying to do with NFTs.”