Author Archives: Pascal Deschenes

NuBot 3.0 on the starting blocks: what’s new!

Because our latest version of our NuBot Platform product is on the verge of being released, I’d like to present a few of the noteworthy new features out in this release.

Enhanced Composite States

Composite states now feature full-fledged encapsulation. In our previous installment, because of the limitations of the transition model, composite states had to be aware of their parent to function properly, greatly limiting the ability to reuse callflow components in some specific cases. Composite states now give more freedom in this regard, thanks to our new transition model, which gives you the ability to break down a transition into two parts, where one part would be defined in the parent, and the other part in the child.

UI Cosmetic Improvement

As part of our effort to offer the best user experience, we have improved a few UI items which should make your life easier. For instance, any section content available from the Results view now features a copy to clipboard button. Also, the Callflow editor now provides the ability to add comments to the layout or move transition labels while offering a more pleasant color scheme. We have also removed some under-used or plainly obtrusive features which did not add any value to the experience.

Callflow Anchor Points and new Transition Model

Our callflow transition model has been refined and now features the ability to break down a given transition into two distinct parts. This characteristic gives you the ability to not only lay out your callflow more easily by routing transitions around states but also enhance your Composite States encapsulation.

Augmented Test Descriptor Editor

We have moved some of the configuration parameters off the Preferences to the Test Descriptor Editor itself, where it makes more sense. All test configuration parameters are now neatly organized into one single location. Such parameters include Call Profile and Call Sampling.

REST Communication Protocol

In order to facilitate both customer firewall and forward proxy traversal, we got rid of the RMI protocol in favor of a friendlier HTTP one. While from a user perspective, this change does not bring much to the table, thanks to a nicely encapsulated communication layer, this protocol overhaul opens the door for a plethora of new integration schemes. For one, you can now use our HTTP API to not only tap into our test call data set but also interact with the platform itself, launching a given test at a specific time from the command line, or PUT/GET test resources on/off the server. The API can also be used to monitor your minutes usage or simply to poll for server status.

# get all scheduled test
curl -X GET -H "X-nuecho-access-key: foobar" \
-H "X-nuecho-secret-key: $4$02Y4dYz+$d61BHuJq/GqylW0p6jVzs1/Arxs$" \
'https://nubot.nuecho.com/api/v0.1/foobar/operations/scheduled'

# delete resource bucket
curl -X DELETE -H "X-nuecho-access-key: foobar" \
-H "X-nuecho-secret-key: $4$02Y4dYz+$d61BHuJq/GqylW0p6jVzs1/Arxs$" \
'https://nubot.nuecho.com/api/v0.1/foobar/resources/3868dd95-a81e-4480-aca6-fa05012075ff'

# get stats from audio service
curl -X GET -H "X-nuecho-access-key: foobar" \
-H "X-nuecho-secret-key: $4$02Y4dYz+$d61BHuJq/GqylW0p6jVzs1/Arxs$" \
'https://nubot.nuecho.com/api/v0.1/foobar/audio/stats'

# launch a test session
curl -X PUT -H "X-nuecho-access-key: foobar" \
-H "X-nuecho-secret-key: $4$02Y4dYz+$d61BHuJq/GqylW0p6jVzs1/Arxs$" \
'https://nubot.nuecho.com/api/v0.1/foobar/operations/launched/3868dd95-a81e-4480-aca6-fa05012075ff?recording=false'

Performance Optimizations

The platform has been re-architectured to handle larger call volumes. Internal services can now be horizontally scaled for high throughput and greater performance. More than ever, you can use our NuBot platform to not only perform regression or functional testing but also to launch much larger load or stress tests.

Conclusion

I hope you are as thrilled as we are about this upcoming release and that you will see benefits from this sneak preview! I intend to present some of those new features in separated posts in the coming weeks.

IVR Application Monitoring: What For?

I’ve recently blogged about what any good IVR monitoring service should provide in terms of reports. Now, let me take a step back, and address some of the reasons why you might want to consider monitoring your IVR application in the first place.

Customer Satisfaction

Satisfaction

Photo Credits: Sanja Gjenero

Customer satisfaction should be one of your primary objectives, as always. As part of any good customer satisfaction strategy, you certainly want to have the overall user experience as smooth as possible. While a smooth experience relies on an efficient dialog interaction design, following industry best-practices, continuous tuning, along with a high level of testing, all these efforts become irrelevant the minute your customers call your application and get a busy ring tone, dead-air, latencies, or plain transaction failure.

Our Mirador service continuously monitors your application, calling in and performing transactions with your IVR system, exactly as a real customer would. As soon as one of the selected performance metrics does not meet the configured level, a real-time alarm notification is sent to your operations team, which can instantly take action and therefore limit any potential negative consequences. Instant notification, instant reaction.

Service Level Agreement (SLA)

Contract

Photo Credits: shho

To quote wikipedia,

“A service level agreement (frequently abbreviated as SLA) is a part of a service contract where the level of service is formally defined. In practice, the term SLA is sometimes used to refer to the contracted delivery time (of the service) or performance.” – Wikipedia

Here, performance is the key. And performance is tied not just to the IVR application itself, but to the overall infrastructure. That is, you not only have to make sure that your application meets the performance requirements set by your internal stakeholders, but you also need to be able to assess that your infrastructure providers meet their respective SLA. When you think infrastructure providers, think about the whole solution in terms of point of failures from a customer perspective, including telecommunication pipes, toll-free and local numbers (all of them!), telephony hardware and software, IVR and CTI platforms, databases, etc.

But then again, how can you assess that such requirements are met? Metrics to the rescue! Continuously gathering metrics about the health of the overall application is key to such SLA acceptance or even to back your claim about any unacceptable performance issues from your provider.

Proactive Provisioning

Grocery Cart

Photo Credits: akaak19

You have launched your IVR service a while ago and you’re really satisfied in terms of ROI. However, in the last few months, you have heard about customers starting to complain about latencies. How odd. This situation could be explained easily: your IVR service has gained popularity. While your overall infrastructure was initially provisioned to comfortably handle a maximum of 10K calls on peak periods, your system is now handling 25K calls, thanks to an efficient marketing and communication department! While analyzing Erlang tables for your specific requirements, there is unfortunately no actual way to achieve prescience and predict your overall system popularity and success.

The only way to avoid user experience degradation due to usage growth is to rely on proactive provisioning. By continuously monitoring your application, gathering performance metrics over time gives you a powerful way to see trends and anticipate problems that will unavoidably occur if the infrastructure is not appropriately expanded. A constant increase in terms of response delay constitutes a good indicator of call handling performance degradation, while an increase in terms of both call setup time and failures might highlight a problem in terms of telecommunication capacity.

Revenue

Coins

Photo Credits: Zsuzsanna Kilian

Last but not least: Revenue. There is a good chance that your business is interested in a constant revenue stream, where more transactions mean more profits. From this perspective, there is also a good chance that your IVR application serves as one of your company’s income vehicle, where each customer calling might be converted into a profitable transaction, either directly or indirectly. In this regard, each call lost may also mean a lost transaction and hence, potential revenue gone… forever. While a web transaction is, by its redundant nature, more tolerant to failures, where connection can be re-established, and transactions rolled over and such, it is not quite the case for over-the-phone transactions. Once the communication with your customer is gone, it is really gone. There is no such thing as a callback failover, agents or systems calling back your potential-revenue-customer. That could be a interesting business idea though. But that is another story : )

Mirador monitors your IVR system to make sure it is up and running, ready to accept incoming calls and perform transactions. That is, potential revenue-generating transactions.

Conclusion

Can your customers truly reach your IVR application? What about now? …And now? …now?

Coming up next: What’s in your monitoring alarm notification triggers?

Nu Echo Introduces the Mirador IVR Application Monitoring Service

The most effective way to make sure your speech or Touch-Tone IVR systems are up and running and provide the user experience you expect

MONTREAL, QC, February 8, 2011 – Nu Echo, creator of the NuBot Automated IVR Testing Platform, is introducing Mirador, an IVR Monitoring Service that makes sure your speech or Touch-Tone IVR systems are up and running and provide the user experience you expect.

Mirador continuously calls your IVR applications at regular intervals and simulates callers going through various application transactions, providing real-time notifications of performance degradations or system failures, as well as periodic reports detailing the system’s performance over time. Because this is all done remotely from our hosted platform, there is nothing to install at the IVR premises and there is no need to modify the IVR applications.

Read the full version.

What’s in your IVR application monitoring report?

In a recent discussion over Hacker News, someone came up with a request for an IVR application monitoring service, suggesting that this is something which should be rather easy to build. Indeed, the dialing is rather easy. A few hacks with Tropo, Twilio or some custom Asterisk scripts would do the trick, but keep in mind that such monitoring service should interact with the IVR the same way a user would (but that’s another story and an upcoming blog post!).

However, as I have pointed out myself, it is one thing to periodically call a given number, it is another to send daily, weekly, monthly and yearly reports to reflect the actual state of the IVR application over time.

Moreover, those reports needs to provide insightful and reliable information. That’s where Mirador comes handy.

Stability Metrics

Mirador Report - Stability Metrics

First and foremost, your report should give a quick overview of the overall stability of your IVR application over a given time period (daily, weekly, monthly, or yearly). Such metrics essentially provide the overall success rate of your application, where setup failures could be caused by various telephony/network errors such as timeout, busy or congestion, while transaction failures are errors occurring once the connection is established.

Performance Metrics

Mirador Report - Performance Metrics 1


Next, we have some performance metrics, which include average call duration, setup time, transaction duration and greeting delay for both all and successful calls. This raw data is also used to depict an interesting performance over time chart, where one can visually spot specific time periods.

While most data can be gathered quite simply, the greeting delay is totally different beast. It corresponds to the actual delay to get the initial application prompt following a successful call setup, as a user would feel it. To compute such data, we used a few interesting speech recognition tricks of ours :)

Mirador Report - Timing Distributions

How do you know whether a user is waiting 1s or 10s for your application to answer? Or, when a user is supposed to take 2 minutes to complete a given transaction or task, how do you know if that is really the case? Performance metrics would not be complete without some distribution charts to highlight such information. To get a better understanding of how well your IVR application responds to some peak periods in production, we have crafted two distribution charts which not only depict setup times but also transaction durations.

Alarm History

Any serious monitoring service should provide email or SMS notification whenever a defect occurs (otherwise, what’s the point of monitoring?). Mirador can be configured to act upon certain thresholds or specific criteria and send alarm notifications right away, in real-time. While alarm occurrence is one thing, alarm restore is another. Indeed, you not only want to know whenever a problem occurred but also the moment the situation has been acted upon and restored.

Mirador Report - Alarm History

That is why a good monitoring report should present a list of all the alarms for a given time period!

Call Detail Records

Mirador Report - Call Detail Records

Lastly, but not the least: the ability to review call detail records (CDRs). Especially those generating alarms. You might want to know when such calls occurred, what was their actual status, duration and so on. You might even be interested in listening to the complete call recordings while you are at it.

Conclusion

Reports are an integral part of any monitoring service. Plus, you certainly would like to review them within your email client, online in a secure location or as a PDF document, to share with your peers. Ideally, you would have a web dashboard where you could access report history, setup new monitoring configurations, reschedule a configuration, define alarm thresholds and notification targets, and so on.

Mirador - PDF Email Web

Mirador - PDF Email Web

Mirador IVR application monitoring service  features all of the previously mentioned characteristics, except for the dashboard. But we are working on it so stay tuned for more!

So, what’s in your IVR application monitoring report?

NuBot IVR Application Testing Platform Beta 2 is now live!

We are proud to announce that the NuBot Testing Platform Beta 2 is now live and being used by the first beta program participants. As a reminder, during the NuBot Beta 2 Program, the NuBot Platform is available free of charge as a SaaS solution through the NuBot Hosting Service.

Automated testing should be considered a critical component of any rigorous development process. The NuBot Platform has been designed to address key limitations of existing IVR testing solutions. With NuBot:

  • No programming skills are required to build test scenarios
  • Numerous and varied test scenarios can be developed quickly and easily, even for large and complex applications
  • Maintenance of test scenarios is facilitated thanks to built-in robustness to application changes
  • A fully integrated suite of analysis tools are available to facilitate diagnosis of test failures.

To learn more about the NuBot Automated IVR Application Testing Platform or to enlist as a Beta Program participant, please contact us.