Spectra - Transcoding / Storage / Object Storage Issue – Incident details

All systems operational

Transcoding / Storage / Object Storage Issue

Resolved
Operational
Started over 1 year agoLasted less than a minute

Affected

Hosting Infrastructure

Operational from 1:52 AM to 1:52 AM

Web Server

Operational from 1:52 AM to 1:52 AM

Object Storage

Operational from 1:52 AM to 1:52 AM

Database

Operational from 1:52 AM to 1:52 AM

Video Processing

Operational from 1:52 AM to 1:52 AM

Remote Runners

Operational from 1:52 AM to 1:52 AM

Updates
  • Resolved
    UTC
    Resolved

    After a LOT of investigation, we finally figured out the root cause. It turns out, the server was silently failing on transferring files of a certain size over to our Object Storage. We couldn't easily debug it, because it gave very little context. However, we managed to fix a long-standing issue, and the clog of old videos that got stuck are all being published with their original datestamp values.

    We apologize for any inconvenience, but we should be way more stable moving forward!