Tag Archives: mirador

IVR Application Monitoring: What For?

I’ve recently blogged about what any good IVR monitoring service should provide in terms of reports. Now, let me take a step back, and address some of the reasons why you might want to consider monitoring your IVR application in the first place.

Customer Satisfaction

Satisfaction

Photo Credits: Sanja Gjenero

Customer satisfaction should be one of your primary objectives, as always. As part of any good customer satisfaction strategy, you certainly want to have the overall user experience as smooth as possible. While a smooth experience relies on an efficient dialog interaction design, following industry best-practices, continuous tuning, along with a high level of testing, all these efforts become irrelevant the minute your customers call your application and get a busy ring tone, dead-air, latencies, or plain transaction failure.

Our Mirador service continuously monitors your application, calling in and performing transactions with your IVR system, exactly as a real customer would. As soon as one of the selected performance metrics does not meet the configured level, a real-time alarm notification is sent to your operations team, which can instantly take action and therefore limit any potential negative consequences. Instant notification, instant reaction.

Service Level Agreement (SLA)

Contract

Photo Credits: shho

To quote wikipedia,

“A service level agreement (frequently abbreviated as SLA) is a part of a service contract where the level of service is formally defined. In practice, the term SLA is sometimes used to refer to the contracted delivery time (of the service) or performance.” – Wikipedia

Here, performance is the key. And performance is tied not just to the IVR application itself, but to the overall infrastructure. That is, you not only have to make sure that your application meets the performance requirements set by your internal stakeholders, but you also need to be able to assess that your infrastructure providers meet their respective SLA. When you think infrastructure providers, think about the whole solution in terms of point of failures from a customer perspective, including telecommunication pipes, toll-free and local numbers (all of them!), telephony hardware and software, IVR and CTI platforms, databases, etc.

But then again, how can you assess that such requirements are met? Metrics to the rescue! Continuously gathering metrics about the health of the overall application is key to such SLA acceptance or even to back your claim about any unacceptable performance issues from your provider.

Proactive Provisioning

Grocery Cart

Photo Credits: akaak19

You have launched your IVR service a while ago and you’re really satisfied in terms of ROI. However, in the last few months, you have heard about customers starting to complain about latencies. How odd. This situation could be explained easily: your IVR service has gained popularity. While your overall infrastructure was initially provisioned to comfortably handle a maximum of 10K calls on peak periods, your system is now handling 25K calls, thanks to an efficient marketing and communication department! While analyzing Erlang tables for your specific requirements, there is unfortunately no actual way to achieve prescience and predict your overall system popularity and success.

The only way to avoid user experience degradation due to usage growth is to rely on proactive provisioning. By continuously monitoring your application, gathering performance metrics over time gives you a powerful way to see trends and anticipate problems that will unavoidably occur if the infrastructure is not appropriately expanded. A constant increase in terms of response delay constitutes a good indicator of call handling performance degradation, while an increase in terms of both call setup time and failures might highlight a problem in terms of telecommunication capacity.

Revenue

Coins

Photo Credits: Zsuzsanna Kilian

Last but not least: Revenue. There is a good chance that your business is interested in a constant revenue stream, where more transactions mean more profits. From this perspective, there is also a good chance that your IVR application serves as one of your company’s income vehicle, where each customer calling might be converted into a profitable transaction, either directly or indirectly. In this regard, each call lost may also mean a lost transaction and hence, potential revenue gone… forever. While a web transaction is, by its redundant nature, more tolerant to failures, where connection can be re-established, and transactions rolled over and such, it is not quite the case for over-the-phone transactions. Once the communication with your customer is gone, it is really gone. There is no such thing as a callback failover, agents or systems calling back your potential-revenue-customer. That could be a interesting business idea though. But that is another story : )

Mirador monitors your IVR system to make sure it is up and running, ready to accept incoming calls and perform transactions. That is, potential revenue-generating transactions.

Conclusion

Can your customers truly reach your IVR application? What about now? …And now? …now?

Coming up next: What’s in your monitoring alarm notification triggers?

Nu Echo Introduces the Mirador IVR Application Monitoring Service

The most effective way to make sure your speech or Touch-Tone IVR systems are up and running and provide the user experience you expect

MONTREAL, QC, February 8, 2011 – Nu Echo, creator of the NuBot Automated IVR Testing Platform, is introducing Mirador, an IVR Monitoring Service that makes sure your speech or Touch-Tone IVR systems are up and running and provide the user experience you expect.

Mirador continuously calls your IVR applications at regular intervals and simulates callers going through various application transactions, providing real-time notifications of performance degradations or system failures, as well as periodic reports detailing the system’s performance over time. Because this is all done remotely from our hosted platform, there is nothing to install at the IVR premises and there is no need to modify the IVR applications.

Read the full version.

What’s in your IVR application monitoring report?

In a recent discussion over Hacker News, someone came up with a request for an IVR application monitoring service, suggesting that this is something which should be rather easy to build. Indeed, the dialing is rather easy. A few hacks with Tropo, Twilio or some custom Asterisk scripts would do the trick, but keep in mind that such monitoring service should interact with the IVR the same way a user would (but that’s another story and an upcoming blog post!).

However, as I have pointed out myself, it is one thing to periodically call a given number, it is another to send daily, weekly, monthly and yearly reports to reflect the actual state of the IVR application over time.

Moreover, those reports needs to provide insightful and reliable information. That’s where Mirador comes handy.

Stability Metrics

Mirador Report - Stability Metrics

First and foremost, your report should give a quick overview of the overall stability of your IVR application over a given time period (daily, weekly, monthly, or yearly). Such metrics essentially provide the overall success rate of your application, where setup failures could be caused by various telephony/network errors such as timeout, busy or congestion, while transaction failures are errors occurring once the connection is established.

Performance Metrics

Mirador Report - Performance Metrics 1


Next, we have some performance metrics, which include average call duration, setup time, transaction duration and greeting delay for both all and successful calls. This raw data is also used to depict an interesting performance over time chart, where one can visually spot specific time periods.

While most data can be gathered quite simply, the greeting delay is totally different beast. It corresponds to the actual delay to get the initial application prompt following a successful call setup, as a user would feel it. To compute such data, we used a few interesting speech recognition tricks of ours :)

Mirador Report - Timing Distributions

How do you know whether a user is waiting 1s or 10s for your application to answer? Or, when a user is supposed to take 2 minutes to complete a given transaction or task, how do you know if that is really the case? Performance metrics would not be complete without some distribution charts to highlight such information. To get a better understanding of how well your IVR application responds to some peak periods in production, we have crafted two distribution charts which not only depict setup times but also transaction durations.

Alarm History

Any serious monitoring service should provide email or SMS notification whenever a defect occurs (otherwise, what’s the point of monitoring?). Mirador can be configured to act upon certain thresholds or specific criteria and send alarm notifications right away, in real-time. While alarm occurrence is one thing, alarm restore is another. Indeed, you not only want to know whenever a problem occurred but also the moment the situation has been acted upon and restored.

Mirador Report - Alarm History

That is why a good monitoring report should present a list of all the alarms for a given time period!

Call Detail Records

Mirador Report - Call Detail Records

Lastly, but not the least: the ability to review call detail records (CDRs). Especially those generating alarms. You might want to know when such calls occurred, what was their actual status, duration and so on. You might even be interested in listening to the complete call recordings while you are at it.

Conclusion

Reports are an integral part of any monitoring service. Plus, you certainly would like to review them within your email client, online in a secure location or as a PDF document, to share with your peers. Ideally, you would have a web dashboard where you could access report history, setup new monitoring configurations, reschedule a configuration, define alarm thresholds and notification targets, and so on.

Mirador - PDF Email Web

Mirador - PDF Email Web

Mirador IVR application monitoring service  features all of the previously mentioned characteristics, except for the dashboard. But we are working on it so stay tuned for more!

So, what’s in your IVR application monitoring report?