Spectra - Degraded Performance - Object Storage – Incident details

All systems operational

Degraded Performance - Object Storage

Resolved
Partial outage
Started about 2 years agoLasted about 2 hours

Affected

Hosting Infrastructure

Partial outage from 11:59 PM to 1:02 AM, Degraded performance from 11:59 PM to 1:02 AM, Operational from 11:59 PM to 2:14 AM, Partial outage from 1:02 AM to 1:12 AM, Degraded performance from 1:02 AM to 1:12 AM, Partial outage from 1:12 AM to 2:14 AM, Degraded performance from 1:12 AM to 2:14 AM

Web Server

Partial outage from 11:59 PM to 2:14 AM

Object Storage

Degraded performance from 11:59 PM to 2:14 AM

Database

Operational from 11:59 PM to 2:14 AM

Updates
  • Resolved
    UTC
    Resolved

    It looks like most videos should work just fine with uploading and storage now, but we do still have a few videos stuck in limbo. We'll be reaching out after a few hours to check in.

  • Monitoring
    UTC
    Monitoring

    Our upload / transcode / store test passed, now we're just monitoring an existing stuck job to see if we can get it to follow suit.

  • Update
    UTC
    Update

    We are running some tests to see whether the culprit has been isolated or not. Stay tuned!

  • Investigating
    UTC
    Investigating

    Some of our videos don't seem to be getting successfully uploaded to our Object Storage provider. We're currently investigating why this is happening, and trying to fix up a couple of stuck jobs. It's possible that something changed in the integration when we moved over that needs to be accounted for.