In a recent discussion over Hacker News, someone came up with a request for an IVR application monitoring service, suggesting that this is something which should be rather easy to build. Indeed, the dialing is rather easy. A few hacks with Tropo, Twilio or some custom Asterisk scripts would do the trick, but keep in mind that such monitoring service should interact with the IVR the same way a user would (but that’s another story and an upcoming blog post!).
However, as I have pointed out myself, it is one thing to periodically call a given number, it is another to send daily, weekly, monthly and yearly reports to reflect the actual state of the IVR application over time.
Moreover, those reports needs to provide insightful and reliable information. That’s where Mirador comes handy.
Stability Metrics

Mirador Report - Stability Metrics
First and foremost, your report should give a quick overview of the overall stability of your IVR application over a given time period (daily, weekly, monthly, or yearly). Such metrics essentially provide the overall success rate of your application, where setup failures could be caused by various telephony/network errors such as timeout, busy or congestion, while transaction failures are errors occurring once the connection is established.
Performance Metrics

Mirador Report - Performance Metrics 1
Next, we have some performance metrics, which include average call duration, setup time, transaction duration and greeting delay for both all and successful calls. This raw data is also used to depict an interesting performance over time chart, where one can visually spot specific time periods.
While most data can be gathered quite simply, the greeting delay is totally different beast. It corresponds to the actual delay to get the initial application prompt following a successful call setup, as a user would feel it. To compute such data, we used a few interesting speech recognition tricks of ours

Mirador Report - Timing Distributions
How do you know whether a user is waiting 1s or 10s for your application to answer? Or, when a user is supposed to take 2 minutes to complete a given transaction or task, how do you know if that is really the case? Performance metrics would not be complete without some distribution charts to highlight such information. To get a better understanding of how well your IVR application responds to some peak periods in production, we have crafted two distribution charts which not only depict setup times but also transaction durations.
Alarm History
Any serious monitoring service should provide email or SMS notification whenever a defect occurs (otherwise, what’s the point of monitoring?). Mirador can be configured to act upon certain thresholds or specific criteria and send alarm notifications right away, in real-time. While alarm occurrence is one thing, alarm restore is another. Indeed, you not only want to know whenever a problem occurred but also the moment the situation has been acted upon and restored.

Mirador Report - Alarm History
That is why a good monitoring report should present a list of all the alarms for a given time period!
Call Detail Records

Mirador Report - Call Detail Records
Lastly, but not the least: the ability to review call detail records (CDRs). Especially those generating alarms. You might want to know when such calls occurred, what was their actual status, duration and so on. You might even be interested in listening to the complete call recordings while you are at it.
Conclusion
Reports are an integral part of any monitoring service. Plus, you certainly would like to review them within your email client, online in a secure location or as a PDF document, to share with your peers. Ideally, you would have a web dashboard where you could access report history, setup new monitoring configurations, reschedule a configuration, define alarm thresholds and notification targets, and so on.

Mirador - PDF Email Web
Mirador IVR application monitoring service features all of the previously mentioned characteristics, except for the dashboard. But we are working on it so stay tuned for more!
So, what’s in your IVR application monitoring report?