Grammars

Dragonfly’s core is a language object model containing the following objects:

• Grammars – these represent collections of rules.
• Rules – these implement complete or partial voice commands, and contain a hierarchy of elements.
• Elements – these form the language building blocks of voice commands, and represent literal words, element sequences, references to other rules, etc.

To illustrate this language model, we discuss an example grammar which contains 2 voice commands: “command one” and “(second command | command two) [test]”.

• Grammar: container for the two voice commands

  • Rule: first voice command rule “command one”

    • Literal element: element for the literal words “command one”. This element is the root-element of the first command rule

  • Rule: second voice command rule “(second command | command two) [test]”

    • Sequence element: root-element of the second command rule

      • Alternative element: first child element of the sequence

        • Literal element: element for the literal words “second command”
        • Literal element: element for the literal words “command two”

      • Optional element: second child element of the sequence

        • Literal element: element for the literal words “test”

All of these different objects are described below and in subsections.

Grammar classes

Recognition callbacks

The speech recognition engine processes the audio it receives and calls the following methods of grammar classes to notify them of the results:

• Grammar.process_begin(): Called when the engine detects the start of a phrase, e.g. when the user starts to speak. This method checks the grammar’s context and activates or deactivates its rules depending on whether the context matches.
• Grammar._process_begin(): Called by Grammar.process_begin() allowing derived classes to easily implement custom functionality without losing the context matching implemented in Grammar.process_begin().
• Grammar.process_recognition(): Called when recognition has completed successfully and results are meant for this grammar. If defined, this method should return whether to continue rule processing afterwards (True or False).
• Grammar.process_recognition_other(): Called when recognition has completed successfully, but the results are not meant for this grammar.
• Grammar.process_recognition_failure(): Called when recognition was not successful, e.g. the microphone picked up background noise.

The last three methods are not defined for the base Grammar class. They are only called if they are defined for derived classes.

Example Grammar using recognition callbacks

from dragonfly import Grammar

class CallbackGrammar(Grammar):

    def process_recognition(self, words, results):
        print("process_recognition()")
        print(words)
        print(results)

        # Grammar rule processing should continue after this method.
        return True

    def process_recognition_other(self, words, results):
        print("process_recognition_other()")
        print(words)
        print(results)

    def process_recognition_failure(self, results):
        print("process_recognition_failure()")
        print(results)

Recognition callbacks with results objects

The last three methods mentioned above can define an optional results parameter, the value of which differs between each SR engine back-end:

SR engine back-end	Type of results objects
Dragon/Natlink	ResObj [*]
Kaldi	Recognition [†]
CMU Pocket Sphinx	None
WSR/SAPI 5	ISpeechRecoResultDispatch [‡] [§]
Text input (“text”)	None

[*]	See the natlink.txt file for info on ResObj.

[†]	See Kaldi documentation section.

[‡]	See the SAPI 5 documentation on ISpeechRecoResultDispatch for how to use it.

[§]	SAPI 5 does not yield results objects for other grammars, so process_recognition_other() callbacks will return None instead.

Grammar class

class Grammar(name, description=None, context=None, engine=None)

Grammar class for managing a set of rules.

This base grammar class takes care of the communication between Dragonfly’s object model and the backend speech recognition engine. This includes compiling rules and elements, loading them, activating and deactivating them, and unloading them. It may, depending on the engine, also include receiving recognition results and dispatching them to the appropriate rule.

• name – name of this grammar
• description (str, default: None) – description for this grammar
• context (Context, default: None) – context within which to be active. If None, the grammar will always be active.

_process_begin(executable, title, handle)

Start of phrase callback.

This usually is the method which should be overridden to give derived grammar classes custom behavior.

This method is called when the speech recognition engine detects that the user has begun to speak a phrase. This method is called by the Grammar.process_begin method only if this grammar’s context matches positively.

Arguments:

• executable – the full path to the module whose window is currently in the foreground.
• title – window title of the foreground window.
• handle – window handle to the foreground window.

activate_rule(rule)

Activate a rule loaded in this grammar.

Internal: this method is normally not called directly by the user, but instead automatically when the rule itself is activated by the user.

active_rules

List of a grammar’s active rules.

add_all_dependencies()

Iterate through the grammar’s rules and add all the necessary dependencies.

Internal This method is called when the grammar is loaded.

add_dependency(dep)

Add a rule or list dependency to this grammar.

Internal: this method is normally not called by the user, but instead automatically during grammar compilation.

add_list(lst)

Add a list to this grammar.

Lists cannot be added to grammars that are currently loaded.

Parameters:lst (ListBase) – Dragonfly list

add_rule(rule)

Add a rule to this grammar.

The following rules apply when adding rules into grammars:

1. Rules cannot be added to grammars that are currently loaded.
2. Two or more rules with the same name are not allowed.

Warning

Note that while adding the same Rule object to more than one grammar is allowed, it is not recommended! This is because the context and active/enabled states of these rules will not function correctly if used. It is better to use separate Rule instances for each grammar instead.

Parameters:rule (Rule) – Dragonfly rule

context

A grammar’s context, under which it and its rules will be active and receive recognitions if it is also enabled.

deactivate_rule(rule)

Deactivate a rule loaded in this grammar.

Internal: this method is normally not called directly by the user, but instead automatically when the rule itself is deactivated by the user.

disable()

Disable this grammar so that it is not active to receive recognitions.

enable()

Enable this grammar so that it is active to receive recognitions.

enabled

Whether a grammar is active to receive recognitions or not.

engine

A grammar’s SR engine.

enter_context()

Enter context callback.

This method is called when a phrase-start has been detected. It is only called if this grammar’s context previously did not match but now does match positively.

exit_context()

Exit context callback.

This method is called when a phrase-start has been detected. It is only called if this grammar’s context previously did match but now doesn’t match positively anymore.

get_complexity_string()

Build and return a human-readable text giving insight into the complexity of this grammar.

lists

List of a grammar’s lists.

load()

Load this grammar into its SR engine.

loaded

Whether a grammar is loaded into its SR engine or not.

name

A grammar’s name.

process_begin(executable, title, handle)

Start of phrase callback.

Usually derived grammar classes override ``Grammar._process_begin`` instead of this method, because this method merely wraps that method adding context matching.

This method is called when the speech recognition engine detects that the user has begun to speak a phrase.

Arguments:

• executable – the full path to the module whose window is currently in the foreground.
• title – window title of the foreground window.
• handle – window handle to the foreground window.

remove_list(lst)

Remove a list from this grammar.

Lists cannot be removed from grammars that are currently loaded.

Parameters:lst (ListBase) – Dragonfly list

remove_rule(rule)

Remove a rule from this grammar.

Rules cannot be removed from grammars that are currently loaded.

Parameters:rule (Rule) – Dragonfly rule

rule_names

List of grammar’s rule names.

rules

List of a grammar’s rules.

set_context(context)

Set the context for this grammar, under which it and its rules will be active and receive recognitions if it is also enabled.

Use of this method overwrites any previous context.

Contexts can be modified at any time, but will only be checked when process_begin() is called.

Parameters:context (Context|None) – context within which to be active. If None, the grammar will always be active.

set_exclusive(exclusive)

Alias of set_exclusiveness().

set_exclusiveness(exclusive)

Set the exclusiveness of this grammar.

unload()

Unload this grammar from its SR engine.

update_list(lst)

Update a list’s content loaded in this grammar.

Internal: this method is normally not called directly by the user, but instead automatically when the list itself is modified by the user.

ConnectionGrammar class

class ConnectionGrammar(name, description=None, context=None, app_name=None)

Grammar class for maintaining a COM connection well within a given context. This is useful for controlling applications through COM while they are in the foreground. This grammar class will take care of dispatching the correct COM interface when the application comes to the foreground, and releasing it when the application is no longer there.

• name – name of this grammar.
• description – description for this grammar.
• context – context within which to maintain the COM connection.
• app_name – COM name to dispatch.

application

COM handle to the application.

connection_down()

Method called immediately after exiting this instance’s context and disconnecting from the application.

By default this method doesn’t do anything. This method should be overridden by derived classes if they need to clean up after disconnection.

connection_up()

Method called immediately after entering this instance’s context and successfully setting up its connection.

By default this method doesn’t do anything. This method should be overridden by derived classes if they need to synchronize some internal state with the application. The COM connection is available through the self.application attribute.