Opbeat Applications Study Highlights Error Timing Concerns

A new study (downloadable as a PDF) by DevOps Platform Opbeat shows that a majority of Web and mobile application errors occur outside normal working hours. This creates negative experiences for end users who may not be able to get the support they need when they need it, which can lead them to switch apps before a developer’s support desk reopens the next day.

Opbeat is a performance monitoring tool for application developers that tracks response times, monitors feature releases and logs errors.

Developers can use the Opbeat Intake API as part of their extended application deployment workflow to send performance-related information to the Opbeat platform.

Opbeat has been able to aggregate data from its service, tracking more than 10 million errors and 40,000 application releases and updates from its platform’s international user base.

73% of Errors Happen Outside Working Hours

Analyzing these findings, Opbeat has released the Opbeat Application Error Study. The study reveals what happens following some of the standard deployment processes that engineering teams may take for granted in both startup and small-business development culture.

One of the mainstays of dev culture is that new releases are usually shipped after lunch, and never on a Friday or a weekend. This is usually because mornings are spent making final tweaks before shipping or fixing other application bugs and Error Handling that came in overnight. That allows the dev team to be available to track the new release in the afternoon and correct any errors that are an expected part of new code deployments.

But while the goal may be to ensure that hands are on deck just after a release has made it into the wild, the result is that the majority of errors experienced by users occur outside of a 9-to-5 working day. Overall, Opbeat’s study demonstrates that 73% of all application errors being logged by its users occur outside of working hours.

While error rates were marginally higher immediately following new releases and at times of high traffic, errors were being logged fairly continuously at other times, albeit at a lower rate.

There was a definite correlation between new releases and error rates, potentially signaling that new releases do mean higher rate of application errors. This will need to be analyzed a bit more, as it could be because factors like reduced traffic over the weekends are also leading to reduced error rates.

24/7 Support or Lose the Customer

The business impact of the study is significant. A study by AppDynamics last year found that more than 80% of app users delete apps after performance issues. That study found that a third of app users (28% in the U.K. and 38% in the U.S.) would switch to another app. Early last year at API World, Jay Srinivasan from Appurify showed that 83% of users won’t open an app again after two crashes.

The findings also point to one of the Resource challenges that businesses with mobile and Web app products need to consider. One of the benefits of open (public) APIs has been that they can help businesses operate in a global marketplace. A company based in Norway could provide a seamless service to customers in Australia, despite the time difference. By creating automatic registration for APIs, for example, some API providers are seeing developers register for an API Key, test the API and put forward a business case to their company for Integration. This could be done overnight, whereas previously the API Request may have sat in an email inbox until the following morning, and even then faced further delays while the company considered how best to onboard the potential new customer. While API providers may have solved this problem through automating processes, application developers must now solve a similar challenge.

APIs to the Rescue

Here again, APIs may be the answer. Twilio offers some in-built integrations that, for example, automatically send SMS texts from Zendesk customer service complaints directly to a dev engineer’s mobile, much like how an on-call doctor would be notified of an emergency. API aggregation service Zapier and others also offer integration tools to help provide dev teams with an around-the-clock notification system that sends alerts for application errors, but these are often triggered by an end-user complaint. In light of the new Opbeat study, more such integrations will be needed, perhaps directly from DevOps tools like Opbeat, where the error itself, rather than an end user’s frustrated support desk email, triggers an alert.

Be sure to read the next Application Development article: Daily API RoundUp: Google Container Engine, Neocities, CardStreams