Tag Archives: test

Ymor’s Ymonitor Performance and Availability Service Now Leverages the NuBot IVR Testing Platform

Nu Echo and Ymor join forces to provide a state of the art automated IVR monitoring solution to the Netherlands and Europe

Ymor, the specialist in creative and innovative performance measurement methods focusing on the BENELUX market, and Nu Echo, a specialist in speech-enabled call center application development, testing, and tuning tools, announce the integration of the NuBot IVR Testing Platform into the Ymonitor Performance and Availability service.

“Integrating the NuBot Platform allows Ymor to support the Dutch market to a level unmatched by any other solution available today,” says Richard Budding, Co-Founder and Managing Director at Ymor. “By working closely with Nu Echo, we have been able to enhance an already feature-rich solution to meet our customer needs and requirements.”

Key benefits obtained by Ymor as a result of integrating the NuBot Platform include the ability to not only gather performance and availability metrics from IVR systems, but also to aggregate continuous data into their monitoring dashboard, through a flexible yet powerful web API. Such capabilities were previously unavailable in the Dutch market.

“We are very pleased to see our solution integrated into Ymor’s offering,” says Yves Normandin, Co-Founder and CEO of Nu Echo. “Our close collaboration with Ymor has enriched our testing and monitoring solutions, while making them available to enterprise and telecommunications organizations in the Dutch market.”

About Nu Echo
Nu Echo develops high performance speech recognition applications, as well as development, testing, and tuning products. Building on two decades of expertise in speech recognition application development, Nu Echo is uniquely equipped to deliver applications, services and product that simply work better. The company also provides a full range of professional services, including grammar development and tuning, speech application testing, and consulting. For more information, please visit http://www.nuecho.com.

About Ymor
Ymor is a pragmatic company that has built up a unique position in the field of performance management. Its focus areas include validation, monitoring, tuning and controlling of complex ICT-areas. Ymor’s expertise delivers added value for its customers, which is essential during the application life cycle. Ymor is a partner of Precise, a software producer of advanced performance management solutions. For more information please visit the company website on http://www.ymor.nl

Original Press Release

NuBot 3.0 on the starting blocks: what’s new!

Because our latest version of our NuBot Platform product is on the verge of being released, I’d like to present a few of the noteworthy new features out in this release.

Enhanced Composite States

Composite states now feature full-fledged encapsulation. In our previous installment, because of the limitations of the transition model, composite states had to be aware of their parent to function properly, greatly limiting the ability to reuse callflow components in some specific cases. Composite states now give more freedom in this regard, thanks to our new transition model, which gives you the ability to break down a transition into two parts, where one part would be defined in the parent, and the other part in the child.

UI Cosmetic Improvement

As part of our effort to offer the best user experience, we have improved a few UI items which should make your life easier. For instance, any section content available from the Results view now features a copy to clipboard button. Also, the Callflow editor now provides the ability to add comments to the layout or move transition labels while offering a more pleasant color scheme. We have also removed some under-used or plainly obtrusive features which did not add any value to the experience.

Callflow Anchor Points and new Transition Model

Our callflow transition model has been refined and now features the ability to break down a given transition into two distinct parts. This characteristic gives you the ability to not only lay out your callflow more easily by routing transitions around states but also enhance your Composite States encapsulation.

Augmented Test Descriptor Editor

We have moved some of the configuration parameters off the Preferences to the Test Descriptor Editor itself, where it makes more sense. All test configuration parameters are now neatly organized into one single location. Such parameters include Call Profile and Call Sampling.

REST Communication Protocol

In order to facilitate both customer firewall and forward proxy traversal, we got rid of the RMI protocol in favor of a friendlier HTTP one. While from a user perspective, this change does not bring much to the table, thanks to a nicely encapsulated communication layer, this protocol overhaul opens the door for a plethora of new integration schemes. For one, you can now use our HTTP API to not only tap into our test call data set but also interact with the platform itself, launching a given test at a specific time from the command line, or PUT/GET test resources on/off the server. The API can also be used to monitor your minutes usage or simply to poll for server status.

# get all scheduled test
curl -X GET -H "X-nuecho-access-key: foobar" \
-H "X-nuecho-secret-key: $4$02Y4dYz+$d61BHuJq/GqylW0p6jVzs1/Arxs$" \
'https://nubot.nuecho.com/api/v0.1/foobar/operations/scheduled'

# delete resource bucket
curl -X DELETE -H "X-nuecho-access-key: foobar" \
-H "X-nuecho-secret-key: $4$02Y4dYz+$d61BHuJq/GqylW0p6jVzs1/Arxs$" \
'https://nubot.nuecho.com/api/v0.1/foobar/resources/3868dd95-a81e-4480-aca6-fa05012075ff'

# get stats from audio service
curl -X GET -H "X-nuecho-access-key: foobar" \
-H "X-nuecho-secret-key: $4$02Y4dYz+$d61BHuJq/GqylW0p6jVzs1/Arxs$" \
'https://nubot.nuecho.com/api/v0.1/foobar/audio/stats'

# launch a test session
curl -X PUT -H "X-nuecho-access-key: foobar" \
-H "X-nuecho-secret-key: $4$02Y4dYz+$d61BHuJq/GqylW0p6jVzs1/Arxs$" \
'https://nubot.nuecho.com/api/v0.1/foobar/operations/launched/3868dd95-a81e-4480-aca6-fa05012075ff?recording=false'

Performance Optimizations

The platform has been re-architectured to handle larger call volumes. Internal services can now be horizontally scaled for high throughput and greater performance. More than ever, you can use our NuBot platform to not only perform regression or functional testing but also to launch much larger load or stress tests.

Conclusion

I hope you are as thrilled as we are about this upcoming release and that you will see benefits from this sneak preview! I intend to present some of those new features in separated posts in the coming weeks.

More robust automated test scripts: wraparound mode

Lately, I have been involved in the development of a new reusable VoiceXML dialog module. The module is invoked via a <subdialog> call with a number of parameters, one of which having an impact on the order of the questions asked by the module.

Writing automated test scripts for such parameterized applications or modules is too often a very time-consuming task. One has to take the order of questions into account, leading to an explosion in the number of scenarios and lots of duplication. In such cases, you often end up testing a single configuration, assuming that all others will be only small variations that need not be tested. But is it really safe to do that?

One of the nice features of NuBot is the ability to write test scenarios that are robust to the order in which questions are asked. To do that, test scenarios need only be created in wraparound mode. Each scenario is composed of action groups, each of which consists in an association between a state in the application and an answer to give to the tested application.

In the wraparound mode, when NuBot receives a feedback from the application, it looks at its next group. If the feedback does not match the expected action group, instead of generating an error, it simply skips it and considers the next one, and so on. If it reaches the end of the scenario’s groups, it “wraps around” (thus the mode name) and considers the groups from the start of the scenario in turn. Only if it cannot match a step in the scenario will it generate an error.

IVR Application Monitoring: What For?

I’ve recently blogged about what any good IVR monitoring service should provide in terms of reports. Now, let me take a step back, and address some of the reasons why you might want to consider monitoring your IVR application in the first place.

Customer Satisfaction

Satisfaction

Photo Credits: Sanja Gjenero

Customer satisfaction should be one of your primary objectives, as always. As part of any good customer satisfaction strategy, you certainly want to have the overall user experience as smooth as possible. While a smooth experience relies on an efficient dialog interaction design, following industry best-practices, continuous tuning, along with a high level of testing, all these efforts become irrelevant the minute your customers call your application and get a busy ring tone, dead-air, latencies, or plain transaction failure.

Our Mirador service continuously monitors your application, calling in and performing transactions with your IVR system, exactly as a real customer would. As soon as one of the selected performance metrics does not meet the configured level, a real-time alarm notification is sent to your operations team, which can instantly take action and therefore limit any potential negative consequences. Instant notification, instant reaction.

Service Level Agreement (SLA)

Contract

Photo Credits: shho

To quote wikipedia,

“A service level agreement (frequently abbreviated as SLA) is a part of a service contract where the level of service is formally defined. In practice, the term SLA is sometimes used to refer to the contracted delivery time (of the service) or performance.” – Wikipedia

Here, performance is the key. And performance is tied not just to the IVR application itself, but to the overall infrastructure. That is, you not only have to make sure that your application meets the performance requirements set by your internal stakeholders, but you also need to be able to assess that your infrastructure providers meet their respective SLA. When you think infrastructure providers, think about the whole solution in terms of point of failures from a customer perspective, including telecommunication pipes, toll-free and local numbers (all of them!), telephony hardware and software, IVR and CTI platforms, databases, etc.

But then again, how can you assess that such requirements are met? Metrics to the rescue! Continuously gathering metrics about the health of the overall application is key to such SLA acceptance or even to back your claim about any unacceptable performance issues from your provider.

Proactive Provisioning

Grocery Cart

Photo Credits: akaak19

You have launched your IVR service a while ago and you’re really satisfied in terms of ROI. However, in the last few months, you have heard about customers starting to complain about latencies. How odd. This situation could be explained easily: your IVR service has gained popularity. While your overall infrastructure was initially provisioned to comfortably handle a maximum of 10K calls on peak periods, your system is now handling 25K calls, thanks to an efficient marketing and communication department! While analyzing Erlang tables for your specific requirements, there is unfortunately no actual way to achieve prescience and predict your overall system popularity and success.

The only way to avoid user experience degradation due to usage growth is to rely on proactive provisioning. By continuously monitoring your application, gathering performance metrics over time gives you a powerful way to see trends and anticipate problems that will unavoidably occur if the infrastructure is not appropriately expanded. A constant increase in terms of response delay constitutes a good indicator of call handling performance degradation, while an increase in terms of both call setup time and failures might highlight a problem in terms of telecommunication capacity.

Revenue

Coins

Photo Credits: Zsuzsanna Kilian

Last but not least: Revenue. There is a good chance that your business is interested in a constant revenue stream, where more transactions mean more profits. From this perspective, there is also a good chance that your IVR application serves as one of your company’s income vehicle, where each customer calling might be converted into a profitable transaction, either directly or indirectly. In this regard, each call lost may also mean a lost transaction and hence, potential revenue gone… forever. While a web transaction is, by its redundant nature, more tolerant to failures, where connection can be re-established, and transactions rolled over and such, it is not quite the case for over-the-phone transactions. Once the communication with your customer is gone, it is really gone. There is no such thing as a callback failover, agents or systems calling back your potential-revenue-customer. That could be a interesting business idea though. But that is another story : )

Mirador monitors your IVR system to make sure it is up and running, ready to accept incoming calls and perform transactions. That is, potential revenue-generating transactions.

Conclusion

Can your customers truly reach your IVR application? What about now? …And now? …now?

Coming up next: What’s in your monitoring alarm notification triggers?

Nu Echo Introduces the Mirador IVR Application Monitoring Service

The most effective way to make sure your speech or Touch-Tone IVR systems are up and running and provide the user experience you expect

MONTREAL, QC, February 8, 2011 – Nu Echo, creator of the NuBot Automated IVR Testing Platform, is introducing Mirador, an IVR Monitoring Service that makes sure your speech or Touch-Tone IVR systems are up and running and provide the user experience you expect.

Mirador continuously calls your IVR applications at regular intervals and simulates callers going through various application transactions, providing real-time notifications of performance degradations or system failures, as well as periodic reports detailing the system’s performance over time. Because this is all done remotely from our hosted platform, there is nothing to install at the IVR premises and there is no need to modify the IVR applications.

Read the full version.