Cardano stake pool operator (SPO) and users have reported at least half of the network nodes to be offline over the weekend.
According to a January 22 post shared on the SPO Telegram for Global Input Output, engineering and fintech research behind the Cardano blockchain, the anomaly caused 50% of Cardano nodes to disconnect and restart.
Last night during the anomaly in the #Cardano network, all networks are not down. There is a short period of degradation. Most of the affected nodes have recovered well. No need to restart the network. pic.twitter.com/FupQXk2otV
– Rick McCracken DIGI (@RichardMcCrackn) January 22, 2023
“This appears to have been triggered by a temporary anomaly that caused two reactions on the node, some disconnecting from peers, others throwing exceptions and restarting,” the post read when explaining the sudden outage.
Despite the momentary degradation, the Cardano network recovered without external intervention. As explained in the post, “these temporary issues” were taken into account in the design of nodes and consensus and “the system behaves as expected.”
During the anomaly, which occurred between blocks 8300569 and 8300570, block production reportedly continued to run but slowed for several minutes and “had little impact, similar to the delays that occur during normal operations.”
“Most nodes automatically recover, depending on which SPO you choose.”
At the time of writing, the root cause of the anomaly and the resulting node disconnection and restart is still under investigation.
“We are currently investigating the root cause of this anomalous behavior and implementing more logging measures along with regular monitoring procedures,” the official announcement said.
related: Cardano will launch a new algorithmic stablecoin in 2023
Tom Stokes, co-founder of Node Shark and Cardano SPO, reported in a January 22 post that more than half of the registered nodes are affected.
He also shared a graph showing where network sync dropped from 100% to just above 40% for over 300 reporting nodes.
A few hours ago over half of everything #Cardano the node goes offline. This is why decentralization is important. pic.twitter.com/NXDVyKy8ep
– Tom Stokes (@eUTxO_pro) January 22, 2023
According to the Stokes graph, network synchronization recovered to 87% after the drop but did not immediately return to its previous level at 100%.
Another SPO reported the same problem for Stoke on January 22 post, but declared “Some SPOs do not see the impact.”
“Others have relays and BP restarts. SPOs, Devs, and IOG are on Discord debugging atm. No cause,” he said.