Sifflet - Downtime on eu-west-1-a cell – Incident details

Downtime on eu-west-1-a cell

Resolved
Major outage
Started 17 days agoLasted about 9 hours

Affected

Web application

Major outage from 7:31 AM to 7:40 AM, Operational from 7:40 AM to 4:49 PM

eu-west-1-a - Web application

Major outage from 7:31 AM to 7:40 AM, Operational from 7:40 AM to 4:49 PM

Integrations

Major outage from 7:31 AM to 7:40 AM, Operational from 7:40 AM to 4:49 PM

eu-west-1-a - Integrations

Major outage from 7:31 AM to 7:40 AM, Operational from 7:40 AM to 4:49 PM

Datasource monitoring

Major outage from 7:31 AM to 7:40 AM, Operational from 7:40 AM to 4:49 PM

eu-west-1-a - Datasource monitoring

Major outage from 7:31 AM to 7:40 AM, Operational from 7:40 AM to 4:49 PM

Updates
  • Resolved
    Resolved

    This incident has been resolved. We updated our monitoring to trigger a page should this condition occurs again, and we identified the root cause of the issue. We're implemeting mitigations to prevent this incident from occuring again.

  • Update
    Update

    We're still performing a complete assessment and postmortem for this incident, but we can share a few preliminary pieces of information.

    Only tenants hosted on the aws-prod-eu-west-1-a cell were affected. The incident started at 9:39 pm CEST on 2025-06-26 and was solved at 9:35 CEST on 2025-06-27. Monitors and source ingestions scheduled during this time did not run. The web application was also unavailable.

    Our monitoring systems were not configured to send a page for the events associated to this incident. As a result, the downtime (outside business hours) was unusually long. We are reviewing our alerting notifications to make sure this doesn't happen again.

    We apologize for the impact caused by this incident. Such a long downtime is not acceptable and we'll implement measures to prevent this from occuring in the future.

  • Monitoring
    Monitoring
    We solved the cause of the downtime. We're monitoring the situation to ensure all services stay available.
  • Investigating
    Investigating
    All tenants in the eu-west-1-a cell are experiencing high error rates. We're investigating.