Resolved -
Service appears to have stabilised after minor modifications and stopping of some request-heavy processes. Documentation has been relocated and updated to reflect recent troubleshooting.
Jun 23, 14:43 BST
Update -
Identified and requested stop of a process scanning gitlab regularly and many times per second
No 500s reported or noted in the logs since 15:19, Gitlab Service appears to be restored
Jun 19, 17:00 BST
Investigating -
We are again receiving reports of impacted performance, and observations of 500 status code errors from the logs
Jun 19, 16:39 BST
Monitoring -
Following the restart, Gitlab appears to be back to normal functionality - currently leaving Debug logs enabled for further monitoring of service performance
Jun 19, 12:37 BST
Investigating -
Rolling out a fresh deployment of the Gitlab and restarting Redis to clear the cache, to hopefully restore normal service while investigation continues
Jun 19, 11:55 BST
Monitoring -
Root Cause as yet unidentified
Jun 19, 11:04 BST
Investigating -
Reports of Gitlab inaccessible giving error 500, as well as automated Gitlab and Drone e2e tests firing repeatedly
Currently investigating root cause of this outage
Jun 18, 14:43 BST