Wiki/Operation/RCAs.md

936 B

These are some recent unplanned downtimes on the AniNIX, starting from 8/24/2016. We provide this list and root-cause analyses (RCA's) so that other networks can learn from them or suggest better practices for us to follow. Past years' RCA's are recorded in :Category:RCA Archive. Category:Operation

{{DowntimeRCA|Charter ISP Outage|cause=ISP|length=4.5 hours|resolution=AniNIX off-site admin detected the outage at 2019-06-18 16:08 CDT. Phone calls to the ISP confirmed a physical outage. ISP acknowledged the outage and field techs were sent to repair the outage in the area. Service was restored at 20:16 CDT.|commits=Our Zapier/Freshping alerting service worked appropriately, and Sharingan sent out notifications as designed when service resumed. However, admin staff had run out of coffee -- coffee will be purchased to prevent further outages, as the tech stops working when the coffee runs out.}}