Post-mortem analysis of data glitch on Aug 3

Following the data glitch that happened on August 3, 2018, we have investigated multiple internal sources to provide an analysis on what happened and what we will do to ensure that a similar incident doesn’t happen again.

What happened on Aug 3, 2018

  • We first started getting alerts from our internal system, notifying us of possible issues on multiple listings. We always actively monitor all incoming alerts so that we can stay ahead of any issues that might have arisen on the site.
IMG_0282
  • We noticed that this data glitch started happening immediately after the team excluded all VEN markets (which was trading at ~$1.61) so that the VET markets would be the markets to be included in the price. At that point in time, there was a long, ongoing swap for VEN to VET, which we attempted to consolidate under one listing to preserve the continuity of the tokens on the site.
  • Due to this exclusion, there was a 90% devaluation of the the VeChain listing.
  • Because of the devaluation of the VeChain markets, the linked Tether markets caused Tether to be valued at $100+ on the new VET markets. With the advent of these $100+ markets, the average price of Tether was skewed to be roughly ~10% higher than normal.
  • Since USDT accounts for ~20% of all reported crypto trades, an inflated Tether price has a huge effect on the prices of most of the major cryptocurrencies. This led to the huge jump in prices reflected on the site.
  • Soon after the discovery of the alerts and glitches on our end, the social media reports started streaming in. We responded quickly to every request for information, across social media, email and tickets, giving a quick explanation for the analysis of Tether markets.
  • Normally, we have safeguards in place where a market reporting $100+ for Tether would be marked as an outlier, but in this case that did not accurately trigger. This is due to a bug in our system which did not deal completely with the edge case of the timing of excluding most or all markets of a cryptocurrency.

What we will do to prevent future occurrences

  1. Going forward, we will fix this bug to ensure that it will accurately mark outliers in the system, especially when we are excluding markets.
  2. We will separate listings in the future, especially with regards to swaps that are not 1:1. We will create a new listing for the new cryptocurrency and keep the old cryptocurrency around until the markets turn to zero volume, and transition accordingly. We will label these appropriately so that it is clear to the user which is which, for example, in this case: VeChain [TOKEN]/[OLD] and VeChain [COIN]/[NEW].
  3. While we do have an internal system in place already alerting us of possible issues, we will continue to improve on this system to ensure that we can be aware of, and respond to, any similar situations in the future more expediently.

We sincerely apologize for this glitch, and thank you for the patience and support that you have shown us in the face of this incident.

Please rest assured that all of us are always watching the alert system carefully and ensuring the integrity of our data at all times. This is the reason why other data providers continue to rely on our deeply-researched circulating supply, price and volume numbers in reporting on their own products.

As always, we are open to listening to your feedback and suggestions for how to continue improving and evolving the CoinMarketCap site. Please send us any other questions or ideas you may have on our form or email newsletter@coinmarketcap.com. Again, thank you for being part of our community and we continue to faithfully serve you with our products.