Natlink and DNS engine back-end

The Dragonfly library may be used with Nuance’s Dragon NaturallySpeaking (DNS) speech recognition software for Microsoft Windows. Dragonfly interacts with DNS through another piece of software called Natlink.

This is the engine of choice on Microsoft Windows.

Requirements and Installation

There are three requirements for using Dragonfly’s Natlink/DNS engine back-end: Dragon NaturallySpeaking for Windows, Natlink and a 32-bit Python installation.

Dragon NaturallySpeaking, or simply Dragon, is available for purchase from Nuance Communications, Inc. For the latest version, see www.nuance.com/dragon.html. Natlink and Dragonfly support Dragon versions up to 16, the latest. The Individual editions of Dragon are recommended, although other editions such as Home will also work. Editions of Dragon for macOS are not compatible with Natlink or Dragonfly.

Natlink is available for download from GitHub and SourceForge. The required version depends on which version of Python is installed. If you are using Python 2, please download and install one of the older versions of Natlink, available from the project’s SourceForge page. If you are using Python 3, please download and install one of the newer versions of Natlink, available from the project’s GitHub releases page.

Please see the Natlink install instructions on qh.antenna.nl for how to install Natlink on your machine and configure it.

A 32-bit Python installation is required to use this engine back-end. This is because Natlink does not work with 64-bit Python. This limitation is unlikely to be overcome.

Post-install Use

Once Natlink is up and running, Dragonfly command-modules can be treated as any other Natlink macro files. Natlink automatically loads macro files from a predefined directory or from the optional user directory. Common locations are:

• C:\NatLink\NatLink\MacroSystem

• C:\Program Files\NatLink\MacroSystem

• My Documents\Natlink

At least one of these, or the custom user directory, should be present after installing and configuring Natlink. That is the place where you should put Dragonfly command-modules so that Natlink will load them. Command-modules should be Python source files starting with an underscore, e.g. _notepad_example.py.

Don’t forget to turn the microphone off and on again after placing new command-modules in the Natlink directory, because otherwise Natlink does not immediately see the new file.

The Natlink/DNS engine may also be used via the command-line interface.

Engine Configuration

This engine can be configured by passing (optional) keyword arguments to the get_engine() function, which passes them to the engine constructor (documented below). For example:

engine = get_engine("natlink",
  retain_dir="natlink_recordings",
)

The engine can also be configured via the command-line interface:

# Initialize the Natlink engine back-end with custom arguments, then load
# command modules and recognize speech.
python -m dragonfly load _*.py --engine natlink --engine-options \
    retain_dir="natlink_recordings"

Engine API

class NatlinkEngine(retain_dir=None)

Speech recognition engine back-end for Natlink and DNS.

Parameters:

retain_dir (str|None) –

directory to save audio data: A .wav file for each utterance, and retain.tsv file with each row listing (wav filename, wav length in seconds, grammar name, rule name, recognized text) as tab separated values.

If this parameter is used in a module loaded by natlinkmain, then the directory will be relative to the Natlink user directory (e.g. MacroSystem).

DictationContainer

alias of NatlinkDictationContainer

apply_threading_fix()

Apply a workaround that permits essentially normal use of threads while Natlink is in operation.

This method emulates natConnect(True) behavior when running under NatSpeak.exe, or when calling connect() with False. The engine calls this method automatically, when appropriate.

connect(bUseThreads=True)

Connect to natlink.

disconnect()

Disconnect from natlink.

mimic(words)

Mimic a recognition of the given words.

Note

This method has a few quirks to be aware of:

1. Mimic is not limited to one element per word as seen with proper nouns from DNS. For example, “Buffalo Bills” can be passed as one word.

2. Mimic can handle extra formatting in DNS built-in commands.

3. Mimic is case sensitive.

set_exclusiveness(grammar, exclusive)

Set the exclusiveness of a grammar.

set_retain_directory(retain_dir)

Set the directory where audio data is saved.

Retaining audio data may be useful for acoustic model training. This is disabled by default.

If a relative path is used and the code is running via natspeak.exe, then the path will be made relative to the Natlink user directory or base directory (e.g. MacroSystem).

Parameters:

retain_dir (string|None) – retain directory path

speak(text)

Speak the given text using text-to-speech.

Dictation container class for Natlink

This class is derived from DictationContainerBase and implements dictation formatting for the Natlink and Dragon NaturallySpeaking engine.

class NatlinkDictationContainer(words, methods)

Container class for dictated words as recognized by the Dictation element for the Natlink and DNS engine.

A dictation container is created by passing it a sequence of words as recognized by the backend SR engine. Each word must be a Unicode string.

Parameters:

• words (sequence-of-unicode) – A sequence of Unicode strings.

• methods (list-of-triples) – Tuples describing string methods to call on the output.

format(spoken_form=False)

Format and return this dictation as a string.

Arguments:

• spoken_form (bool, default: False) – whether to use the spoken form of dictated words in the result instead of the written form. Only has an effect if using the Natlink/DNS engine back-end

Dictation formatting for Natlink

The classes in this module implement dictation formatting for the Natlink and Dragon NaturallySpeaking engine.

class FlagContainer(*names)

Container for a predefined set of boolean flags.

class StateFlags(*names)

Container for keeping state for inter-word formatting.

The flags defined by this class are used by Dragonfly to store formatting state between words as a formatter consumes consecutive words.

class Word(written, spoken, flags)

Word storing written and spoken forms with formatting flags.

class WordFlags(*names)

Container for formatting flags associated with DNS words.

The flags defined by this class are closely related to the formatting information provided by DNS.

class WordFormatter(state=None, parser=None, two_spaces_after_period=False)

Dictation word formatter class.

This is the main class to use for formatting dictation words from DNS.

format_dictation(input_words, spoken_form=False)

Format input words.

class WordParserBase

Base class for parsing dictation results into Word objects.

class WordParserDns10

Dictation results parser for DNS v10 and lower.

DNS v10 and lower provide word formatting information as a 32-bit value returned by natlink.getWordInfo().

class WordParserDns11

Dictation results parser for DNS v11 and higher.

DNS v11 and higher provide word formatting information …

class WordParserFactory

Detects appropriate WordParser class and creates objects

This class calls natlink.getWordInfo() for special words and interprets the result to determine whether words should be parsed using natlink.getWordInfo(), e.g. for DNS v10 and lower, or using “in-line word properties”, e.g. for DNS v11 and higher.

This class performs detection only once per instance and caches the detection result for reuse.

get_parser()

Create an instance of the detective parser class.

Multiplexing interface to the Natlink timer

class NatlinkTimerManager(interval, engine)

Timer manager for the Natlink engine.

This class utilises natlink.setTimerCallback() to ensure that timer functions are called on-time regardless of Dragon’s current status.

Python code run outside of timer functions will be blocked when natlink functions are executing. This is a limitation with Python threads.

engine.connect() must be called before using timers with this manager.

Note: Long-running timers will block dragonfly from processing what was said, so be careful with how you use them!