TrackAbout System Status
All Systems Operational

About This Site

Sometimes things go wrong. On the rare occurrence they do, we want to be as transparent about our status as we can.

So if things aren't entirely green and operational, just know that it's not you. It's us. And we're working on it.

You can use this page to sign up for notifications about upcoming maintenance and real-time alerts about service issues.

Uptime over the past 90 days. View historical uptime.
Production Services Operational
90 days ago
99.99 % uptime
Today
Production Application Web Site
Operational
90 days ago
99.99 % uptime
Today
Production TAMobile 6 Sync Services
Operational
90 days ago
100.0 % uptime
Today
TAMobile 7 iOS
Operational
90 days ago
100.0 % uptime
Today
TAMobile 7 Android Operational
90 days ago
100.0 % uptime
Today
Application-Sent Email Delivery
Operational
90 days ago
100.0 % uptime
Today
Custom Reports and OpenData
Operational
90 days ago
100.0 % uptime
Today
Production SFTP
Operational
90 days ago
100.0 % uptime
Today
Production FTPS
Operational
90 days ago
100.0 % uptime
Today
Customer Test Services Operational
90 days ago
100.0 % uptime
Today
Customer Test TAMobile 6 Sync Services
Operational
90 days ago
100.0 % uptime
Today
Customer Test Application Web Site
Operational
90 days ago
100.0 % uptime
Today
Marketing Operational
90 days ago
100.0 % uptime
Today
Corporate Web Site
Operational
90 days ago
100.0 % uptime
Today
Customer Support Operational
90 days ago
100.0 % uptime
Today
Inbound Phone Service
Operational
90 days ago
100.0 % uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.

No incidents or maintenance related to this downtime.

System Metrics Month Week Day
Production Uptime (via Pingdom) ?
Fetching
Past Incidents
Sep 23, 2023

No incidents reported today.

Sep 22, 2023

No incidents reported.

Sep 21, 2023

No incidents reported.

Sep 20, 2023

No incidents reported.

Sep 19, 2023

No incidents reported.

Sep 18, 2023

No incidents reported.

Sep 17, 2023

No incidents reported.

Sep 16, 2023

No incidents reported.

Sep 15, 2023

No incidents reported.

Sep 14, 2023
Browser Login Issues on September 14, 2023 with error NS_ERROR_NET_INADEQUATE_SECURITY
Resolved - On Thursday, September 14th, 2023 at approximately 4:12 PM, we received word from a single customer having login problems with the TrackAbout web site (but not the mobile apps). Their browser was reporting an error of type "NS_ERROR_NET_INADEQUATE_SECURITY" when attempting to log in.

After investigating, we could find no cause for such an error. No TrackAbout infrastructure changes had been made that day. We were left to conclude that this must be a local issue with the customer's infrastructure, most likely an HTTP proxy issue.

Several hours later, we received the same complaint from a second customer. With two disparate customers having the same issue, we turned our eyes to our new identity provider Auth0 (Okta) and opened a support ticket.

After some hours, Auth0 responded that they *had* made an infrastructure change without notification and that they had reverted the change. Their change was the cause of our customers' authentication issues.

This morning we received the following post-mortem report from Auth0:

----
"I want to start by apologizing for the impact this had on you and your clients, I can certainly understand why this would put you in a challenging position and I'm sorry that occurred. Our Engineering team provided a summary of the event that I've copied below:

On September 15, 2023, our Engineering team began to investigate a series of TLS cipher negotiation failures impacting our customers across multiple Private Cloud environments as well as our Public Cloud US region. Specifically, a subset of browsers began to experience errors when attempting to use our services through a custom domain.

The root cause of this issue was traced back to certain clients negotiating a cipher with our edge provider, then being rejected due to the cipher being on a banned list.

No further change is required by our customers and we do not expect this issue to recur following the rollback performed by our Engineering team. We sincerely apologize for any impact this had on you and your users.

I can add that the change was made to mitigate a separate certificate related issue that our Engineering team had identified as being a potential problem in the future. This is all of the information that has been provided about this event so far and I hope it's helpful for you and your team. Again I'm sorry for the position this put you in with your clients and please let me know if you have any follow up questions after reviewing."
----

As a preventative measure, we are adding change monitoring of the TLS/SSL certificates protecting our Auth0 authentication URLs. We need to know as quickly as possible if any change has been made to the configuration that Auth0 is managing. We are writing our own monitoring test as well as employing a popular TLS/SSL site quality checker from Qualsys SSL Labs.

Additionally, Auth0 is a relatively new third-party dependency for us, and a critical one at that, as it is the gateway through which all users authenticate.

Going forward, if we receive word of users having login problems, we will immediately open a ticket with Auth0.

Finally, I have let Auth0 know that making unannounced changes involving endpoint security has the potential to put their customers in an uncomfortable position and have strongly recommended that they publish notifications of changes.

Larry Silverman
Chief Technology Officer
TrackAbout, Inc.
Sep 14, 16:00 EDT
Sep 13, 2023

No incidents reported.

Sep 12, 2023

No incidents reported.

Sep 11, 2023

No incidents reported.

Sep 10, 2023

No incidents reported.

Sep 9, 2023

No incidents reported.

← Incident History Powered by Atlassian Statuspage