*If the form isn’t loading or submitting properly, please 1) disable ad blockers, 2) enable cookies, and 3) turn off any tracking protection features. Still can’t see the form? Reach out to sales@bugsnag.com.
User experiences with web and mobile applications are more important than
ever.
Users have high expectations for performance and demand consistent updates with new features and
content.
Thanks to low switching costs and a growing plethora of options available on the market, if users
see even
the smallest issue with stability, speed, or responsiveness, they’ll explore other options.
Stability: A barometer of quality and quantity
Stability
measures
application health as well as user experience. It’s calculated in two main ways to provide
insight. A
percentage of
- Crash-free application sessions (or unhandled exceptions)
- Daily active users who do not experience an error
Our insights help you
determine application stability SLAs and SLOs, like how infrastructure and operational teams rely on
the
“five nines” to measure uptime and availability.
99.91%
Median
Target stability scores
best
represent your SLO. This score can be thought of as an aspirational goal to balance fixing bugs with
building
new features. If your stability score is above target stability, you can focus on building new features.
If
lower, get the bugs.
On the other hand, you can compare our critical stability score to your
SLA.
This is the minimum threshold your app should run on to ensure its functionality. If your stability
score
drops below your critical stability threshold, drop everything and focus on fixing bugs immediately.
To help you compare your application to other leading web and mobile apps, BugSnag has
aggregated
data to analyze stability trends and release frequency with industry benchmarks. We welcome you to learn
from
them as well.
Stability is a percentage of app sessions that are
crash-free.
Stability scores are used to gauge app health and user experience.
Measuring App Quality Across Industries
When it comes to
measuring
app quality, stability hits different industries in varied ways. Factors like value of user
interactions,
and volume. Understanding these factors will help engineering organizations develop strategies for
delivering higher quality apps.
How stability turns loyalty into revenue
App Stability by Industry
High Stability
Trendsetters:
Setting the bar while delivering enriching new experiences
- Media and Entertainment has taken off in the past several years when it comes to streaming media. As many newcomers attempt to grow their users, it remains essential for them to provide an optimal, uninterrupted experience. Meanwhile, to retain their massive subscriber base against increased competition, streaming giants seek to strike a balance between delivering a seamless experience and consistently offering new features to enhance their application – across a range of devices.
- Ecommerce continues to be a driving force even as brick and mortar retail attempts to recover to pre-pandemic levels. As worldwide ecommerce revenue expects to climb past $5.5 trillion in 2022, there’s a lot of money in customers’ carts when they arrive at checkout. Giving customers a smooth experience is paramount, as crashes directly impact revenue.
Medium Stability
Middle of the
Pack: Good with the potential to be great
- B2B SaaS, Technology, Finance & Banking, Education, and Telecom all had high stability scores but weren’t quite high enough to meet the standards set by our Trendsetters. B2B SaaS, Technology, Telecom, and Finance & Banking displayed high scores for the second year in a row. Each these industries are involved in significant financial transactions and often are highly regulated. A stable user experience is essential to success, especially when stability translates to higher loyalty and revenue.
- B2C SaaS/Consumer Mobile, Social, Consumer Goods, Travel & Hospitality all have a strong consumer focus. Consumer Goods, Travel & Hospitality and B2C SaaS have financial transactions that must be secure for consumers’ piece of mind and their continued app use.
Low Stability
Run of the
Mill:
Nothing special and easily replaceable
- Gaming, as in previous editions of our index, continues to lag behind other industries in stability. Despite millions of users, especially among mobile games, these users are less loyal and have no qualms about replacing games with others due to next to none switching costs. To retain these users, games often introduce new features, but this increases the likelihood of bugs. This explains why Gaming also had the most score variability out of all the included industries.
- Health & Wellness also scored below the industry-wide mean of our index while demonstrating large variability in scores. These apps still have a way to go to achieve a healthy stability score and should focus on fixing bugs before introducing any new features or capabilities.
Web vs Mobile: A Comparison
Mobile apps continue to become more stable while web
applications show dramatic improvement
In our reports from 2020 and 2021, median mobile app stability score in-creased
from
99.63% to 99.80%. This year, median stability score for mobile apps continued to trend upwards,
increasing to
99.88% in 2022. While mobile apps are becoming more stable, there is still work to be done to achieve
the
“five nines” of stability.
Web-based applications showed surprising gains in their median
stability score. Just two years ago web applications had a median stability score of 99.39%. In 2020 we
mentioned this may be due to monitoring and ad-dressing client-side issues in JavaScript applications
usually
requires more effort than mobile apps. Evidently, there has been significant strides made in adapting
this
older discipline to today’s needs, as web apps scored a median stability of 99.94% in 2022’s report.
While
this is excellent news and shows the success of implementing error handling strategies, there is still
room to
grow to achieve the “five nines,” like mobile apps.
Different platforms have different errors
Our web
applications data includes several front-end development platforms, including Ember, JavaScript,
React, and
Vue. Each of these platforms is a modern, opinionated JavaScript framework built with consideration
for
error handling. Some specific causes of unhandled exceptions in web applications may be an event
handler bug
causing the user interaction to fail, an unhandled promise rejection warning, a bug preventing a
page from
rendering, and more. Mobile applications data includes several mobile development platforms, among
them
Android, iOS, React Native, and Unity. For these applications, stability scores are negatively
impacted by
session-ending events. This means things like crashes and ANRs (Application Not Responding) in
Android,
React Native, and Unity frameworks as well as OOMs (Out of Memory) in iOS applications.
30 Day stability score by platform
Mobile platforms built with specific personas in mind
Android and iOS continue to have high median
stability
scores. One reason for this consistency is the amount of developers specialized in Android and iOS.
Their
well of knowledge not only helps them fix errors, it helps them do so efficiently.
While
their
median stability scores are similar, the range of scores between Android and iOS applications is
noticeably
different. Many Android applications have a stability lower than the platform median while iOS
applications
have a much narrower range. One reason for this discrepancy is because Android has a much larger number of devices – a more open
development
environment. This can make it difficult to test applications and increases exposure to crashes and
errors.
Whereas dedicated iOS development teams have a limited
number of
devices to test their application’s stability on.
React Native is a very popular mobile development
framework
thanks to its ability to simultaneously develop Android and iOS applications. It is especially
helpful for
engineers that may not be familiar with mobile frameworks but have experience with React for web
applications.
Unity is extremely
popular in
the mobile gaming industry, with many of the most popular games developed on the platform. As
primarily a
mobile gaming framework, it’s not surprising Unity showed the same trends out of the platforms
included in
our report. Mobile games often introduce new features rapidly to satisfy their users, which can lead
to more
errors and instability. Feature flags and experiments to introduce new features to a subsegment of
users
will help improve app stability while minimizing the amount of bugs (or risk) introduced to your
application’s entire user base. As an aside, we recently launched support for Unreal Engine
earlier this
year, however we were unable to use the data for this year’s report. We expect to see Unreal
Engine as a
platform listed next year.
The versatility behind modern web application frameworks
JavaScript and React
showed
similar stability, with JavaScript applications just edging out React applications for the highest
stability
score. React did have a narrower stability score distribution than JavaScript, which may be a result
of
JavaScript having a wider range of industries using its framework than React.
Each
web-based
application has its stability measured across multiple browsers. All customers experiencing errors
in Google
Chrome, Mozilla Firefox, Safari, or other browsers are included in the overall stability score of
the
application.
Sometimes errors may be caused by specific browser extensions, which are included
in our
stability score. However, most engineering organizations do not dedicate time and resources to fix
these
bugs because they are not from their application’s code.
As a disclaimer, because the
volume of
Ember and Vue projects was significantly less than JavaScript and React projects, for the purpose of
our
analysis we grouped Ember and Vue projects with JavaScript projects.
Mobile
apps release at almost twice the frequency of web apps
Web and mobile
applications are
shifting their release cadences as consumers frequently demand new features and immediate bug
resolution. We
sifted through our data to analyze the release frequency of the leading applications so you can
benchmark them
against your release cadence.
One notable example is the trend of faster releases for mobile
apps
which update weekly. On average, across all industries mobile apps deliver every four to five days.
Meanwhile,
web applications adhere to the biweekly release frequency, about every ten days. One possible
explanation for
the discrepancy is mobile apps more frequently test new features by practicing progressive delivery with
feature flags, experiments, and A/B tests to retain their users. This is especially true in mobile
gaming
apps.
30 Day release
frequency: web vs. mobile
Release cadences depend on industry type
Release Frequency by Industry
Gaming has the narrowest distribution,
averaging about 6.5 releases a month. This is an increase over the weekly frequency we saw in
last year’s
report. The frequent release schedule may help explain why gaming saw the lowest stability score
in our
report.
Health & Wellness had the widest range of distribution as well as the
second
highest release median. Their second-lowest stability median is proof why there needs to be a
balance
between frequent and infrequent updates, and how measures like target and critical stability
scores can
help find that balance.
All industries besides Media & Entertainment released a
new
version of their application on a weekly basis at minimum. It’s becoming clearer developers are
more
confident in their releases schedules and accelerating new releases through progressive delivery
practices
like feature flags.
Outside of a few industries, each industry displayed a wide
range of
release schedules, with some releasing multiple times a day. This may indicate bugs are
releasing at large
scale and teams need to focus on minimizing risk with new features.
Key Takeaways
Additional resources
Methodology
- The study examined 260 web and mobile apps using BugSnag for error monitoring and stability management to measure and improve app stability and customer experience.
- The 260 web and mobile apps studied were the ones with the highest 30-day session volume, which is indicative of app usage, to include only the leading apps from each industry.
- Data was aggregated and anonymized, eliminating any company and app user identifiable data from the analysis.
