Performance has remained nominal since our intervention. We regenerated the database statistics in the affected database in order to help the query planner choose more optimal plans. We have a regular job that updates statistics, and we'll look into whether any changes might be required there to avoid future incidents.
We are closing this incident. If we determine an actionable root cause we will follow up with a post-mortem.
Posted Jul 25, 2024 - 11:55 EDT
Monitoring
We identified that one customer was sending in excess of 500 requests per minute, which caused a database performance spike. We took action and the load is now back to normal. We are monitoring and will investigate further the root cause.
Posted Jul 25, 2024 - 10:54 EDT
Identified
We are troubleshooting a performance degradation within the database environment. One database has spiked CPU to 100% and is impacting the performance of the environment.
Posted Jul 25, 2024 - 10:23 EDT
This incident affected: Production Services (Production Application Web Site, Production TAMobile 6 Sync Services, TAMobile 7 iOS, TAMobile 7 Android).