In one of my previous posts, I presented the concept of IVR unit testing. Although a very nice concept in theory, I am sure many of you said to yourself: “Great, but I can’t do that since I use a graphical service creation environment (SCE)”. This may be true for some SCEs, but certainly not for all.
There are a few SCEs, like Cisco Unified Call Studio (formerly CVP Studio), that let you extend the environment with Java code. That’s what we did for one of our professional services projects. Let me briefly explain what we did in this project and present some of the benefits as well as a few lessons learned.
The application
The application was a very typical IVR DTMF-only hierarchical menu: lots of options, many optional messages at various places in the menu tree triggered by dynamic configuration options, information messages, etc. Each menu had to support a number of common navigation commands, like * to repeat, # to go back to the previous menu, and so on. The difficulty with such an application is that duplicating the dialog for each menu is quite time consuming and highly error-prone in the presence of constantly evolving customer requirements.
At least, CVP Studio provides a way to define reusable dialog patterns, but unfortunately once the pattern is copied at various places in your application, it cannot be modified in such a way that all its uses are automatically updated. You have to modify each use of the pattern manually.
In addition to those reusable dialog patterns, CVP Studio provides a programmatic API to implement custom elements. These elements can then be added to the SCE’s palette and used to implement nodes at various places in the application, each with its own configuration. Typically, such custom elements simply add some elements to a VoiceXML page (when they implement the VoiceElementBase interface).
For our application, we implemented all the menu nodes using a custom element. The element encapsulates the common behaviors shared by all menus, like how to handle no input, errors, the repeat key, etc. A key advantage of doing this is that when we need to change one of those behaviors, all the nodes in the application are updated at once (saving us a lot of maintenance headaches). Another advantage is that this custom element can be easily reused in other applications as well.
Of course, some will note that we could have used VoiceXML subdialogs rather than custom elements to implement our reusable dialogs. However, due to the design of our Java-based management console interface to configure all the dynamic elements of the application, it was more natural for us to build custom elements also in Java.
But the coolest thing about this approach is that the whole dialog for a single menu is driven by a small state machine that generates objects representing interactions with the caller (instead of plain XML elements) and accepting objects representing interaction results (like a no input, a no match, a recognition result, a DTMF input, etc.). And we ensured that it is possible to interact with the state machine without having to execute the dialog at run time. The state machines are completely decoupled from CVP Studio’s programmatic API!
Guess what? We could unit test all the menus very easily. We just wrote a test controller that injects interaction results programmatically into the state machine, retrieves the next interaction, and asserts some properties (which prompts are played, what options are available, etc.). It is thus very easy to test all the different situations that can be encountered at run time (call center open or closed, optional message activated or not, and so on).
Here is an example of a simple unit test:
The most interesting lines in this method are the ones near the end, beginning with testCase.addInputAssertion. They tell the test case which answers (interaction) from the user to simulate, as well as some assertions that must hold after the application has processed the interaction. For example, the first call simulates a NO MATCH event and the test case will make sure that the next step from the application will be to play a prompt (message) identified by the constant MenuConfiguration.NO_MATCH_1. The next one simulates a no input event and asserts that the generic options are enabled, and the menu prompts will be played. Finally, the third one simulates another no input event and ensures the call will be transferred.
This example only illustrates the testing of a simple generic behavior. The more interesting test cases involve specific nodes in the call-flow depending heavily on the dynamic configuration of the application. By stubbing the clock, for instance, we can make sure that messages telling the contact center is closed are properly played outside of business hours and that the option to transfer the call to an agent is disabled. Other tests ensure that during business hours, the proper transfer reasons are set before transferring to the call center queue manager.
Some lessons learned
Note that there are also some drawbacks to this approach as well. First, this technique does not make it possible to test the sequencing of these customm nodes in the application. For that, we had to rely on manual testing. But that was not such a big deal after all. The way those custom nodes are connected in CVP makes the validation process quite trivial by comparing the call-flow design document with the call-flow in CVP Studio. For instance, if the application goes from a A to B when DTMF 8 is pressed in the former, there is a transition labelled “8″ from custom node A to custom node B in the latter.
Reporting was another issue we faced due to this approach. CVP already provides extensive reporting capabilities when the application uses only predefined elements. When using custom elements, you have to carefully log events in a special table of the Informix DB, and this greatly complicates the consolidation of information to get a precise understanding of what’s happening with the calls.
Was it worth it?
All in all, the advantages of this approach far outweighed the issues just mentioned. At least for this project, in which a large part of the configuration is dynamic and requires a fair amount of Java code anyway. We still maintain the application and it evolves quite rapidly, even several years after its initial deployment.
Hey, I don’t use CVP!
You don’t use CVP? There are other approaches giving you some of these benefits. I’ll outline some of them in upcoming posts. Stay tuned!
And of course, if you have experience implementing unit testing using graphical service creation environments, please share it with us!
