Resolved -
This incident has been resolved.
May 15, 17:30 BST
Monitoring -
A fix has been implemented and we are monitoring results.
May 14, 15:19 BST
Investigating -
From a recent status update, we identified that Drone CI issues of failing pipelines was resolved via an implemented fix. It has come to our attention that pipelines have began failing once again. We are investigating this issue and apologise for any inconveniences caused.
May 14, 13:51 BST
Resolved -
The issue with Drone pipelines failing has been resolved. New nodes spun up by the cluster autoscaler were missing a required compatibility layer which caused all pipeline steps to fail. The fix has been applied to all affected nodes and pipelines are running successfully again. We are working on a permanent fix to ensure new nodes have this automatically going forward. We apologise for any disruption caused.
May 14, 12:01 BST
Update -
We are continuing to investigate this issue.
May 14, 11:11 BST
Investigating -
We are currently investigating an issue where Drone pipelines are failing due to cluster autoscaling issues. As a result, pipeline executions will refuse to run and there will be potential delays in build and deploy processes.
Further updates will be shared once we identify a resolution.
May 14, 10:11 BST