As you certainly know by reading this blog, our main focus at Nu Echo is on developing the best possible speech applications and tools. That’s why the NuGram Platform is primarily targeting speech application developers.
Our vision, however, extends beyond just speech applications. The NuGram Platform can also be used for text-based applications – for instance SMS or IM-based applications – or even desktop applications.
An example of this was recently provided by researchers at the University of Saarland, Germany. They are using the NuGram Platform for a very interesting application: The Virtual Cocktail Guide. Markus Dräger, who wrote the NL interface, describes the project as follows:
Our interactive system is called ‘Virtual Cocktail Guide’ (VCG), and it is designed to convey information about cocktails in an entertaining and playful way. In doing so, users of the system will also be given the opportunity to prepare actual cocktails. The system consists of a number of different physical objects, i.e., a shaker, different liquor bottles, ice cubes and any other ingredients needed to prepare different cocktails. The “face” of the system is an animated virtual character called Sam, who greets new potential users upon their arrival and gives an overview of the basic functionality of the system.
When the user chooses to prepare a certain cocktail, Sam will engage them in a dialogue, interactively providing instructions, help and/or further information during every single step of the preparation.
The part of the system that I am working on in particular is the component that checks whether the user follows the instruction given by the system, e.g., whether they provide the right amount of a given ingredient. The first step in doing so is to obtain this kind of information from the cocktail recipes that are pre-parsed from online cocktail websites and stored in written natural language. To this end, I have written a grammar that is able to parse those cocktail recipes and returns semantic information via SISR-Tags. For example, the sentence “Please pour 2cl of vodka into the shaker” will result in a semantic structure looking like this: {action:"pour",ingredient:"vodka", target:"shaker", amount:"2cl"}. This semantic information is fed straight into a rule engine, as are the actions performed by the user. The engine then does some reasoning over the action prescribed by the recipe and the actual action of the user, and deduces whether they match completely or partially (or even not at all). Based on the result of the reasoning process, the system then can react with an appropriate dialogue move.
Grammar development, publishing and the semantic interpretation are -of course- accomplished with the help of NuGram IDE and the free hosted server, which both did a great job in our application so far.
It’s always nice to see people using our tools in totally innovative, unexpected ways.
Markus Dräger can be reached by email at “markus.draeger AT stud.uni-saarland.de”. The Virtual Cocktail Guide website describing the whole research project will be launched soon. I’ll post a link here when it will be online.
