Vexera - Elevated errors rate – Incident details

All systems operational

Elevated errors rate

Resolved
Minor outage
Started about 4 years agoLasted about 6 hours

Affected

Bot

Minor outage from 5:00 PM to 10:58 PM

Commands processing

Minor outage from 5:00 PM to 10:58 PM

Updates
  • Resolved
    UTC
    Resolved

    This issue has been resolved.

  • Monitoring
    UTC
    Monitoring

    All affected shards have been restarted. Vexera is fully operational. We'll keep monitoring the situation for another few minutes until we mark the issue resolved.

  • Update
    UTC
    Update

    The recovery process is taking longer than expected due to a huge amount of shards (~700 out of 1535) not having the correct guild count. We are still in the process of restarting all affected shards. Current ETA is 21:45 (UTC). Thank you for your continued patience while we are working towards a resolution to this issue.

  • Identified
    UTC
    Identified

    We have located the root cause of the issue. Recovery is currently in progress, we hope to be back within the next hour. Thank you for your patience!

  • Investigating
    UTC
    Investigating

    Our monitoring system fired an alert regarding elevated errors rate. We are investigating!