Confusability Input and Output File Format

The Confusability tool accepts as input an XML-based grammar file and a configuration file for a recognition engine. You can optionally specify the identifier for a rule in the grammar, or provide a list of phrases to analyze in a separate XML document. The tool performs the confusability analysis on the entire grammar, or on a rule within the grammar (if specified), or on a separate list of phrases (if supplied). The tool writes a summary of the analysis to the console and generates complete results of the analysis to an XML file.

Input File Format

Confusability accepts the following file formats for input grammar files:

• Grammar documents in XML format that conform to the Speech Recognition Grammar Specification (SRGS) Version 1.0, for example “MyGrammar.grxml”. It is an accepted convention to use the “.grxml” file extension for XML-based grammar documents that conform to the SRGS specification.

• Compiled grammar files in CFG format, for example “MyCompiledGrammar.cfg”. See the Compile Grammar Reference Manual for information about compiling grammar files.

Example Input Grammar File

The following is an example of a valid input grammar document in XML format that contains only the minimum required elements and attributes in the document’s header, and a simple rule in the body of the document.

<?xml version="1.0"?>   
<grammar version="1.0"      
xmlns="http://www.w3.org/2001/06/grammar"      
xml:lang="en-US" root="main">      

   <rule id="main">         
      <one-of>            
         <item> to </item>            
         <item> two </item>         
      </one-of>      
   </rule> 
  
</grammar>

A valid XML-format grammar document consists of a legal header followed by a body consisting of a set of legal rule definitions. The document header ends and the body of the grammar document begins with the first rule element. Grammar files used as input for Confusability can contain optional elements and attributes in the document header, see SRGS Grammar XML Reference (Microsoft.Speech). For more information about elements and attributes in SRGS grammars, see Speech Recognition Grammar Specification Version 1.0.

Specifying a List of Phrases

Using the /PhraseFile option, you can specify a list of phrases on which to perform the confusability analysis. The list of phrases may contain phrases that are not in the grammar. We recommend that you use Phrase Generator to produce the list of phrases for input to the Confusability tool. See Phrase Generator Reference Manual. The following is an example of an input file that contains a list of phrases to check against the grammar shown above. Note that the second phrase is not in the grammar.

<?xml version="1.0" encoding="utf-8"?>
<Scenario>
  <Utterances>
    <Utterance>
      <TranscriptText>to</TranscriptText>
    </Utterance>
    <Utterance>
      <TranscriptText>too</TranscriptText>
    </Utterance>
  </Utterances>
</Scenario>

Output File Format

The Confusability tool writes a summary of the analysis to the console and generates complete results of the analysis to an XML file.

Summary Output to the Console

When the analysis is complete, a summary containing the following information displays in the console:

• The total of all phrases analyzed. Depending on which options were specified on the command line, the total represents either all the phrases in the grammar, or all the phrases from a rule within the grammar, or all the phrases in the separate list of phrases.

• A count of phrases that generated in-grammar false accepts.

• Whether the output file was successfully generated.

The following is an example of the summary output to the console:

    Info: 3 phrases analyzed.

    Info: 2 phrases found with confusability greater than zero.

    Info: Successfully output details to "MyOut.xml"

    Processing Complete: 0 Error(s). 0 Warning(s).

Confusability Output File

The Confusability tool generates detailed results of the analysis to an output file in XML format. The output file contains each input phrase (from the phrase file or generated from the grammar) that has some likelihood of being confused by the recognition engine with another phrase in the grammar. The output file includes the following information for each input phrase:

Example

The following example illustrates the output that the Confusability tool generates using the grammar and phrase file examples shown above as input.

<?xml version="1.0" encoding="utf-8"?>
<Scenario xml:space="preserve">
  <Utterances>
    <Utterance id="1">
      <TranscriptText>to</TranscriptText>
      <TranscriptSemantics ms:typespace="ECMA-262" xmlns:ms="https://www.microsoft.com/xmlns/webreco" xmlns:emma="http://www.w3.org/2003/04/emma" />
      <TranscriptPronunciation>T O</TranscriptPronunciation>
      <RecoResultText>two</RecoResultText>
      <RecoResultSemantics ms:typespace="ECMA-262" xmlns:ms="https://www.microsoft.com/xmlns/webreco" xmlns:emma="http://www.w3.org/2003/04/emma" />
      <RecoResultRuleTree>
        <Rule id="main" uri="file:///C:/temp/grammars/Test.grxml">two</Rule>
      </RecoResultRuleTree>
      <RecoResultPronunciation>T U</RecoResultPronunciation>
      <RecoResultConfidence>0.4486466</RecoResultConfidence>
    </Utterance>
    <Utterance id="2">
      <TranscriptText>too</TranscriptText>
      <RecoResultText>to</RecoResultText>
      <RecoResultSemantics ms:typespace="ECMA-262" xmlns:ms="https://www.microsoft.com/xmlns/webreco" xmlns:emma="http://www.w3.org/2003/04/emma" />
      <RecoResultRuleTree>
        <Rule id="main" uri="file:///C:/temp/grammars/Test.grxml">to</Rule>
      </RecoResultRuleTree>
      <RecoResultPronunciation>T U</RecoResultPronunciation>
      <RecoResultConfidence>1.0000000</RecoResultConfidence>
      <InGrammar>False</InGrammar>
    </Utterance>
    <Utterance id="3">
      <TranscriptText>too</TranscriptText>
      <RecoResultText>two</RecoResultText>
      <RecoResultSemantics ms:typespace="ECMA-262" xmlns:ms="https://www.microsoft.com/xmlns/webreco" xmlns:emma="http://www.w3.org/2003/04/emma" />
      <RecoResultRuleTree>
        <Rule id="main" uri="file:///C:/temp/grammars/Test.grxml">two</Rule>
      </RecoResultRuleTree>
      <RecoResultPronunciation>T U</RecoResultPronunciation>
      <RecoResultConfidence>0.4822275</RecoResultConfidence>
      <InGrammar>False</InGrammar>
    </Utterance>
  </Utterances>
</Scenario>

Remarks

It cannot be assumed that the value for RecoResultConfidence will be the same if the confused phrase is used as the input phrase. In the example above, submitting the word "to" returned the word "two" in the first utterance with a RecoResultConfidence of 0.4486466. If we submit "two" as input for analysis, the tool will return "to", but the value for RecoResultConfidence may be different.

Item weights affect the simulated set of results, such that items with low weights are less likely to be a confused result.