Yesterday, Routific was intermittently unavailable for 32 minutes between 10:08am – 10:40am Pacific Time. The incident has been identified and resolved.
What happened?
We deployed a feature that included an inefficient DB query for large datasets. This caused our DB servers to be overloaded, which resulted in very slow performance and intermittently dropped connections. We immediately reverted the deployment as soon as we confirmed that this was indeed the culprit.
What are we doing about it?
Our engineering team is investigating our processes to understand how something like this was able to get into production, despite our current testing and QA practices.
Reliability and stability of the Routific service is of utmost importance to us. We are making sure that we learn from our mistake and work hard towards a more stable platform going forward.
Sorry for the disruption to your business; thanks again for your patience and ongoing support!