The Dragonfly library contains an action framework which offers easy and flexible interfaces to common actions, such as sending keystrokes and emulating speech recognition. Dragonfly’s actions sub-package has various types of these actions, each consisting of a Python class. There is for example a dragonfly.actions.action_key.Key class for sending keystrokes and a dragonfly.actions.action_mimic.Mimic class for emulating speech recognition.
Each of these actions is implemented as a Python class and this makes it easy to work with them. An action can be created (defined what it will do) at one point and executed (do what it was defined to do) later. Actions can be added together with the + operator to attend them together, thereby creating series of actions.
Perhaps the most important method of Dragonfly’s actions is their dragonfly.actions.action_base.ActionBase.execute() method, which performs the actual event associated with its action.
Dragonfly’s action types are derived from the dragonfly.actions.action_base.ActionBase class. This base class implements standard action behavior, such as the ability to concatenate multiple actions and to duplicate an action.
The code below shows the basic usage of Dragonfly action objects. They can be created, combined, executed, etc.
from dragonfly.all import Key, Text a1 = Key("up, left, down, right") # Define action a1. a1.execute() # Send the keystrokes. a2 = Text("Hello world!") # Define action a2, which # will type the text. a2.execute() # Send the keystrokes. a4 = a1 + a2 # a4 is now the concatenation # of a1 and a2. a4.execute() # Send the keystrokes. a3 = Key("a-f, down/25:4") # Press alt-f and then down 4 times # with 25/100 s pause in between. a4 += a3 # a4 is now the concatenation # of a1, a2, and a3. a4.execute() # Send the keystrokes. Key("w-b, right/25:5").execute() # Define and execute together.
Combining voice commands and actions¶
A common use of Dragonfly is to control other applications by voice and to automate common desktop activities. To do this, voice commands can be associated with actions. When the command is spoken, the action is executed. Dragonfly’s action framework allows for easy definition of things to do, such as text input and sending keystrokes. It also allows these things to be dynamically coupled to voice commands, so as to enable the actions to contain dynamic elements from the recognized command.
An example would be a voice command to find some bit of text:
- Command specification: please find <text>
- Associated action: Key("c-f") + Text("%(text)s")
- Special element: Dictation("text")
This triplet would allow the user to say “please find some words”, which would result in control-f being pressed to open the Find dialogue followed by “some words” being typed into the dialog. The special element is necessary to define what the dynamic element “text” is.