Dramatically increased error rates

Incident Report for data.world

Postmortem

At 1:51 pm CDT the data.world site saw an anomalous increase in traffic to a specific set of pages, which caused a brief disruption for all users. As part of our threat detection and response process, at 2:10 pm CDT we blocked a small set of IP addresses responsible for the anomalous traffic to restore normal service.

Further investigation revealed a software bug that was responsible for the site disruption. A fix for the bug was put into place at 2:22 pm CDT and the blocked IPs were unblocked two minutes later.

Most users experienced a 19 minute disruption. Users behind the blocked IPs experienced a 33 minute disruption.

Please note that no data was lost during this incident. We take our availability very seriously and apologize for the inconvenience caused.

Posted Oct 23, 2018 - 09:32 CDT

Resolved

This incident has been resolved.

Posted Oct 23, 2018 - 09:31 CDT

Monitoring

The site is fully operational. We will continue to monitor and are rolling out changes to prevent any recurrence.

Posted Oct 22, 2018 - 15:00 CDT

Identified

We have identified the issue and are in the process of rolling out a fix.

Posted Oct 22, 2018 - 14:30 CDT

Investigating

We are currently investigating an increase in error rates and site unavailability.

Posted Oct 22, 2018 - 13:56 CDT

This incident affected: Website (Uploads, Downloads, Query, Discussions, Activity Feeds).