Monthly Archives: June 2010

An alternative way to automate IVR tests

A few weeks ago, I posted an article describing a real hands-on experience on implementing IVR unit tests in an CVP Studio application. But programmatic unit tests are not the only way to automate IVR application tests and provide a repeatable and reliable way of testing large portions of an application easily, in a matter of minutes. There are other ways to get most of the benefits of unit testing without even having to pick the phone (you have better things to do than make hundreds of phone calls a day, right?).

One of them is NuBot, Nu Echo’s hosted IVR application testing platform. With NuBot, there are basically three steps involved:

  1. You instrument your application with DTMF sequences at specific places in the application’s call-flow. These sequences are used to synchronize the application with your test scenarios and are only played when the application is in test mode. (We also support speech-recognition-based synchronization, but only through our professional services at the moment.)
  2. You program your test scenarios using the free NuBot integrated testing environment (ITE), an Eclipse plugin that can co-exist alongside the rest of your programming environment.
  3. You schedule and launch your test on our hosted platform from the NuBot ITE, specifying which scenarios to use, how many ports are needed, how many runs of each scenario to do, etc.

Now as you modify your application, you simply keep your tests up-to-date and re-run them as needed. They can even be incorporated into an automated continuous integration process running every night. So if you break something in the application, you will know it fast.

Of course, in contrast to unit tests which are run on the developer’s machine, automated tests using NuBot require that the application is deployed on a server first. This requires some extra work. But you would have to do that anyway if you were to do your tests manually. And it’s worth it considering that you are doing end-to-end testing of your application, not just running some Java code.

Load-testing ready

Another advantage of using NuBot is that once your application is instrumented and you have all your test scenarios, they can be readily used for load testing. This way, you won’t have to start planning for the development of load testing scripts only after the application is fully implemented.

And of course, you’ll do the load testing it at your own convenience, all by yourself. This way, you stay in control of your testing process. (We do offer professional services if you prefer, but they are completely optional.)

Try it now!

Want to cut your testing costs while delivering more reliable applications? Give NuBot a try.

We also have an on-premise version if using a hosted platform is not an option. Contact us for more details.

Converting GSL tags to SISR – conflicting goals

NuGram IDE provides a tool to convert Nuance GSL grammars to SRGS ABNF, which can then be converted to XML form. But the tool does not convert the semantic tags. So lately we’ve started working on the conversion of GSL semantic tags to SISR, and what initially seemed like a simple project provoked a heated debate internally (well, I may be exaggerating a bit… ;-) . I soon realized that this was because there are really two competing forces driving the design of such a tool:

  • Correctness.The set of SISR tags generated automatically faithfully implement the behaviour of the corresponding Nuance GSL tags. In other words, the resulting grammar needs no manual intervention and the semantic results obtained using the generated grammar are always the same as if the original GSL grammar was used.
  • Maintenance.The set of SISR tags generated automatically are easy to understand, and thus to modify. They are close to what an SISR developer would have written from the start.

To see why these two goals conflict, simply consider how calls to predefined GSL functions can be translated. The GSL tags language provides predefined functions for things as simple as arithmetic operations: $add, $sub, etc. Converting a GSL tag of the form:

{return (add($n $m))}

could generate a SISR tag like

{out = $add(n,m);}

if we want to preserve correctness. Here $add would be a function defined in a generated grammar header tag that implements an ECMAScript equivalent of the GSL add function, with proper handling of undefined values:

{!{
  function $add(x, y) {
    if (x == undefined || typeof x != "number") x = 0;
    if (y == undefined || typeof y != "number") y = 0;
    return x + y;
  }
}!};

But if the converter inlines the call and adds some code to check for undefined values, it could produce something like:

{out = ((n == undefined || typeof n != "number") ? 0 : n)
        + ((m == undefined || typeof m != "number") ? 0 : m);}

when one would have simply written:

{out = n + m;}

Unfortunately, this last version may produce unexpected results at runtime. If n is undefined and m is 3, the sum will be NaN (not a number) instead of 3 as would have produced the original GSL tag. In that case, it is essential to complement the tool with a rigorous testing process, one that can ideally ensure that each and every semantic tag in the grammar will be executed at least once.

So which one is better?

The answer is: that depends. In some cases correctness is preferable, especially if the grammar requires little to no maintenance at all. That may be true of simple grammars. But most of the time, grammars change over time. New sentence patterns are added, rules are extracted, etc. So maybe it’s best to leave the choice to the developer by offering a flexible tool.

And you, what would you prefer: a conversion tool that ensures full correctness at all costs, or a tool that sometimes produces grammars that are potentially not equivalent to the original one but are more maintainable?