Actions sub-package¶
The Dragonfly library contains an action framework which offers easy and
flexible interfaces to common actions, such as sending keystrokes and
emulating speech recognition. Dragonfly’s actions sub-package has various
types of these actions, each consisting of a Python class. There is for
example a dragonfly.actions.action_key.Key
class for sending
keystrokes and a dragonfly.actions.action_mimic.Mimic
class for
emulating speech recognition.
Each of these actions is implemented as a Python class and this makes it
easy to work with them. An action can be created (defined what it will
do) at one point and executed (do what it was defined to do) later.
Actions can be added together with the +
operator to attend them
together, thereby creating series of actions.
Perhaps the most important method of Dragonfly’s actions is their
dragonfly.actions.action_base.ActionBase.execute()
method, which
performs the actual event associated with its action.
Dragonfly’s action types are derived from the
dragonfly.actions.action_base.ActionBase
class. This base class
implements standard action behavior, such as the ability to concatenate
multiple actions and to duplicate an action.
Basic examples¶
The code below shows the basic usage of Dragonfly action objects. They can be created, combined, executed, etc.
from dragonfly import Key, Text
a1 = Key("up, left, down, right") # Define action a1.
a1.execute() # Send the keystrokes.
a2 = Text("Hello world!") # Define action a2, which
# will type the text.
a2.execute() # Send the keystrokes.
a4 = a1 + a2 # a4 is now the concatenation
# of a1 and a2.
a4.execute() # Send the keystrokes.
a3 = Key("a-f, down/25:4") # Press alt-f and then down 4 times
# with 25/100 s pause in between.
a4 += a3 # a4 is now the concatenation
# of a1, a2, and a3.
a4.execute() # Send the keystrokes.
Key("w-b, right/25:5").execute() # Define and execute together.
More examples¶
For more examples on how to use and manipulate Dragonfly action objects,
please see the doctests for the
dragonfly.actions.action_base.ActionBase
here:
Action doctests.
Combining voice commands and actions¶
A common use of Dragonfly is to control other applications by voice and to automate common desktop activities. To do this, voice commands can be associated with actions. When the command is spoken, the action is executed. Dragonfly’s action framework allows for easy definition of things to do, such as text input and sending keystrokes. It also allows these things to be dynamically coupled to voice commands, so as to enable the actions to contain dynamic elements from the recognized command.
An example would be a voice command to find some bit of text:
- Command specification:
please find <text>
- Associated action:
Key("c-f") + Text("%(text)s")
- Special element:
Dictation("text")
This triplet would allow the user to say “please find some words”, which would result in control-f being pressed to open the Find dialogue followed by “some words” being typed into the dialog. The special element is necessary to define what the dynamic element “text” is.
Action class reference¶
ActionBase base class¶
-
class
ActionSeries
(*actions)[source]¶ -
stop_on_failures
= True¶ Whether to stop executing if an action in the series fails.
-
-
class
Repeat
(extra=None, count=None)[source]¶ Action repeat factor.
Integer Repeat factors ignore any supply data:
>>> integer = Repeat(count=3) >>> integer.factor() 3 >>> integer.factor({"foo": 4}) # Non-related data is ignored. 3
Integer Repeat factors can be specified with the
*
operator:>>> from dragonfly import Function >>> def func(): ... print("executing 'func'") ... >>> action = Function(func) * 3 >>> action.execute() executing 'func' executing 'func' executing 'func'
Named Repeat factors retrieved their factor-value from the supplied data:
>>> named = Repeat("foo") >>> named.factor() Traceback (most recent call last): ... ActionError: No extra repeat factor found for name 'foo' ('NoneType' object is unsubscriptable) >>> named.factor({"foo": 4}) 4
Repeat factors with both integer count and named extra values set combined (add) these together to determine their factor-value:
>>> combined = Repeat(extra="foo", count=3) >>> combined.factor() Traceback (most recent call last): ... ActionError: No extra repeat factor found for name 'foo' ('NoneType' object is unsubscriptable) >>> combined.factor({"foo": 4}) # Combined factors 3 + 4 = 7. 7
Key action¶
This section describes the Key
action object. This
type of action is used for sending keystrokes to the foreground
application. This works on Windows, Mac OS and with X11 (e.g. on Linux).
Examples of how to use this class are given in Example key actions.
To use this class on X11/Linux, the
xdotool program must be
installed and the DISPLAY
environment variable set. This class does
not support typing keys in Wayland sessions.
Keystroke specification format¶
The spec argument passed to the Key
constructor specifies which
keystroke events will be emulated. It is a string consisting of one or
more comma-separated keystroke elements. Each of these elements has one
of the following two possible formats:
- Normal press-release key action, optionally repeated several times:
- [modifiers
-
] keyname [/
innerpause] [:
repeat] [/
outerpause] - Press-and-hold a key, or release a held-down key:
- [modifiers
-
] keyname:
direction [/
outerpause]
The different parts of the keystroke specification are as follows. Note that only keyname is required; the other fields are optional.
modifiers – Modifiers for this keystroke. These keys are held down while pressing the main keystroke. Can be zero or more of the following:
a
– alt keyc
– control keys
– shift keyw
– Windows keykeyname – Name of the keystroke. Valid names are listed in Key names.
innerpause – The time to pause between repetitions of this keystroke. It should be given in hundredths of a second. For example, “20” will pause for 20/100s = 0.2 seconds.
repeat – The number of times this keystroke should be repeated. If not specified, the key will be pressed and released once.
outerpause – The time to pause after this keystroke. It should be given in hundredths of a second. For example, “20” will pause for 20/100s = 0.2 seconds.
direction – Whether to press-and-hold or release the key. Must be one of the following:
down
– press and hold the keyup
– release the keyNote that releasing a key which is not being held down does not cause an error. It harmlessly does nothing.
Key names¶
- Lowercase letter keys:
a
oralpha
,b
orbravo
,c
orcharlie
,d
ordelta
,e
orecho
,f
orfoxtrot
,g
orgolf
,h
orhotel
,i
orindia
,j
orjuliet
,k
orkilo
,l
orlima
,m
ormike
,n
ornovember
,o
oroscar
,p
orpapa
,q
orquebec
,r
orromeo
,s
orsierra
,t
ortango
,u
oruniform
,v
orvictor
,w
orwhisky
,x
orxray
,y
oryankee
,z
orzulu
- Uppercase letter keys:
A
orAlpha
,B
orBravo
,C
orCharlie
,D
orDelta
,E
orEcho
,F
orFoxtrot
,G
orGolf
,H
orHotel
,I
orIndia
,J
orJuliet
,K
orKilo
,L
orLima
,M
orMike
,N
orNovember
,O
orOscar
,P
orPapa
,Q
orQuebec
,R
orRomeo
,S
orSierra
,T
orTango
,U
orUniform
,V
orVictor
,W
orWhisky
,X
orXray
,Y
orYankee
,Z
orZulu
- Number keys:
0
orzero
,1
orone
,2
ortwo
,3
orthree
,4
orfour
,5
orfive
,6
orsix
,7
orseven
,8
oreight
,9
ornine
- Symbol keys:
!
orbang
orexclamation
,@
orat
,#
orhash
,$
ordollar
,%
orpercent
,^
orcaret
,&
orand
orampersand
,*
orstar
orasterisk
,(
orleftparen
orlparen
,)
orrightparen
orrparen
,minus
orhyphen
,_
orunderscore
,+
orplus
,`
orbacktick
,~
ortilde
,[
orleftbracket
orlbracket
,]
orrightbracket
orrbracket
,{
orleftbrace
orlbrace
,}
orrightbrace
orrbrace
,\
orbackslash
,|
orbar
,colon
,;
orsemicolon
,'
orapostrophe
orsinglequote
orsquote
,"
orquote
ordoublequote
ordquote
,comma
,.
ordot
,slash
,<
orlessthan
orleftangle
orlangle
,>
orgreaterthan
orrightangle
orrangle
,?
orquestion
,=
orequal
orequals
- Whitespace and editing keys:
enter
,tab
,space
,backspace
,delete
ordel
- Main modifier keys:
shift
,control
orctrl
,alt
- Right modifier keys:
rshift
,rcontrol
orrctrl
,ralt
- Special keys:
escape
,insert
,pause
,win
,rwin
,apps
orpopup
,snapshot
orprintscreen
- Lock keys:
scrolllock
,numlock
,capslock
- Navigation keys:
up
,down
,left
,right
,pageup
orpgup
,pagedown
orpgdown
,home
,end
- Number pad keys:
npmul
,npadd
,npsep
,npsub
,npdec
,npdiv
,numpad0
ornp0
,numpad1
ornp1
,numpad2
ornp2
,numpad3
ornp3
,numpad4
ornp4
,numpad5
ornp5
,numpad6
ornp6
,numpad7
ornp7
,numpad8
ornp8
,numpad9
ornp9
- Function keys:
f1
,f2
,f3
,f4
,f5
,f6
,f7
,f8
,f9
,f10
,f11
,f12
,f13
,f14
,f15
,f16
,f17
,f18
,f19
,f20
,f21
,f22
,f23
,f24
- Multimedia keys:
volumeup
orvolup
,volumedown
orvoldown
,volumemute
orvolmute
,tracknext
,trackprev
,playpause
,browserback
,browserforward
Example key actions¶
The following code types the text “Hello world!” into the foreground application:
Key("H, e, l, l, o, space, w, o, r, l, d, exclamation").execute()
The following code is a bit more useful, as it saves the current file with the name “dragonfly.txt” (this works for many English-language applications):
action = Key("a-f, a/50") + Text("dragonfly.txt") + Key("enter")
action.execute()
The following code selects the next four lines by holding down the shift key, slowly moving down 4 lines, and then releasing the shift key:
Key("shift:down, down/25:4, shift:up").execute()
The following code locks the screen by pressing the Windows key together with the l key:
Key("w-l").execute()
Windows key support¶
Keyboard events sent by Key
actions on Windows are calculated using
the current foreground window’s keyboard layout. The class will fallback on
Unicode events for keys not typeable with the current layout.
The Key
action can be used to type arbitrary Unicode characters on
Windows using the relevant Windows API.
This is disabled by default because it ignores the up/down status of
modifier keys (e.g. ctrl).
It can be enabled by changing the unicode_keyboard
setting in
~/.dragonfly2-speech/settings.cfg to True
:
unicode_keyboard = True
The use_hardware
parameter can be set to True
if you need to
selectively require hardware events for a Key
action:
# Passing use_hardware=True will guarantee that Ctrl+C is always
# pressed, regardless of the layout. See below.
Key("c-c", use_hardware=True).execute()
If the Unicode keyboard is not enabled or the use_hardware
parameter is
True
, then no keys will be typed and an error will be logged for
untypeable keys:
action.exec (ERROR): Execution failed: Keyboard interface cannot type this character: 'μ'
Keys in ranges 0-9, a-z and A-Z are always typeable. If keys in these ranges cannot be typed using the current keyboard layout, then the equivalent key will be used instead. For example, the following code will result in the ‘я’ key being pressed when using the main Cyrillic keyboard layout:
# This is equivalent to Key(u"я, Я, c-я").
Key("z, Z, c-z", use_hardware=True).execute()
Unlike the Text
action, individual Key
actions can send
both hardware and Unicode events. So the following example will work if
the Unicode keyboard is enabled:
# Type 'σμ' and then press Ctrl+Z.
Key(u"σ, μ, c-z").execute()
X11 key support¶
The Key
action can be used to type arbitrary keys and Unicode
characters on X11/Linux. It is not limited to the key names listed above,
although all of them will work too.
Unicode characters are supported on X11 by passing their Unicode code point
to the keyboard implementation. For example, the character '€'
is
converted to 'U20AC'
. The Unicode code point can also be passed
directly, e.g. with Key('U20AC')
.
Unlike on Windows, the Key
action is able to use modifiers with
Unicode characters on X11.
This class does not support typing keys in Wayland sessions.
Example X11 key actions¶
In addition to the examples in the previous section, the following example will work on X11/Linux.
The following code will type ‘σμ’ into the foreground application and then press ctrl+z:
Key("σ,μ,c-z").execute()
The following code will press ‘µ’ while holding control and alt:
Key("ca-μ").execute()
The following code will press the browser refresh multimedia key:
Key("XF86Refresh").execute()
Although this key is not defined in dragonfly’s typeables list, it still works because it is passed directly to xdotool. X11 (Xorg) multimedia keys can be found online: XF86 keyboard symbols.
Key class reference¶
-
class
Key
(spec=None, static=False, use_hardware=False)[source]¶ Keystroke emulation action.
- Constructor arguments:
- spec (str) – keystroke specification
- static (boolean) – flag indicating whether the specification contains dynamic elements
- use_hardware (boolean) – if True, send keyboard events using hardware emulation instead of as Unicode text. This will respect the up/down status of modifier keys.
The format of the keystroke specification spec is described in Keystroke specification format.
This class emulates keyboard activity by sending keystrokes to the foreground application. It does this using Dragonfly’s keyboard interface for the current platform. The implementation for Windows uses the
sendinput()
Win32 API function. The implementation for Mac OS uses pynput. The implementation for X11/Linux uses xdotool.-
class
EventData
(keyname, direction, modifiers, inner_pause, repeat, outer_pause)¶ Create new instance of EventData(keyname, direction, modifiers, inner_pause, repeat, outer_pause)
-
direction
¶ Alias for field number 1
-
inner_pause
¶ Alias for field number 3
-
keyname
¶ Alias for field number 0
-
modifiers
¶ Alias for field number 2
-
outer_pause
¶ Alias for field number 5
-
repeat
¶ Alias for field number 4
-
Text action¶
This section describes the Text
action object. This type of
action is used for typing text into the foreground application. This works
on Windows, Mac OS and with X11 (e.g. on Linux).
To use this class on X11/Linux, the
xdotool program must be
installed and the DISPLAY
environment variable set. This class does
not support typing text in Wayland sessions.
It differs from the Key
action in that Text
is used for
typing literal text, while dragonfly.actions.action_key.Key
emulates pressing keys on the keyboard. An example of this is that the
arrow-keys are not part of a text and so cannot be typed using the
Text
action, but can be sent by the
dragonfly.actions.action_key.Key
action.
Windows Unicode Keyboard Support¶
The Text
action can be used to type arbitrary Unicode characters
using the relevant Windows API.
This is disabled by default because it ignores the up/down status of
modifier keys (e.g. ctrl).
It can be enabled by changing the unicode_keyboard
setting in
~/.dragonfly2-speech/settings.cfg to True
:
unicode_keyboard = True
If you need to simulate typing arbitrary Unicode characters and have
individual Text
actions respect modifier keys normally for normal
characters, set the configuration as above and use the use_hardware
parameter for Text
as follows:
action = Text("σμ") + Key("ctrl:down") + Text("]", use_hardware=True) + Key("ctrl:up")
action.execute()
Some applications require hardware emulation versus Unicode keyboard
emulation. If you use such applications, add their executable names to the
hardware_apps
list in the configuration file mentioned above to make
dragonfly always use hardware emulation for them.
If hardware emulation is required, then the action will use the keyboard layout of the foreground window when calculating keyboard events. If any of the specified characters are not typeable using the current window’s keyboard layout, then an error will be logged and no keys will be typed:
action.exec (ERROR): Execution failed: Keyboard interface cannot type this character: 'μ'
Keys in ranges 0-9, a-z and A-Z are always typeable. If keys in these ranges cannot be typed using the current keyboard layout, then the equivalent key will be used instead. For example, the following code will result in the ‘я’ key being pressed when using the main Cyrillic keyboard layout:
# This is equivalent to Text(u"яЯ").
Text("zZ").execute()
These settings and parameters have no effect on other platforms.
X11/Linux Unicode Keyboard Support¶
The Text
action can also type arbitrary Unicode characters on X11.
This works regardless of the use_hardware
parameter or
unicode_keyboard
setting.
Unlike on Windows, modifier keys will be respected by Text
actions
on X11. As such, the previous Windows example will work and can even be
simplified a little:
action = Text("σμ") + Key("ctrl:down") + Text("]") + Key("ctrl:up")
action.execute()
It can also be done with one Key
action:
Key("σ,μ,c-]").execute()
Text class reference¶
-
class
Text
(spec=None, static=False, pause=None, autofmt=False, use_hardware=False)[source]¶ Action that sends keyboard events to type text.
- Arguments:
- spec (str) – the text to type
- static (boolean) – if True, do not dynamically interpret spec when executing this action
- pause (float) – the time to pause between each keystroke, given in seconds
- autofmt (boolean) – if True, attempt to format the text with correct spacing and capitalization. This is done by first mimicking a word recognition and then analyzing its spacing and capitalization and applying the same formatting to the text.
- use_hardware (boolean) – if True, send keyboard events using hardware emulation instead of as Unicode text. This will respect the up/down status of modifier keys.
Paste action¶
-
class
Paste
(contents, format=None, paste=None, static=False)[source]¶ Paste-from-clipboard action.
- Constructor arguments:
- contents (str | dict) – contents to paste. This may be a simple string to paste, a dynamic action spec or a dictionary of clipboard format ints to contents (typically strings).
- format (int, clipboard format integer) – clipboard format. This argument is ignored if contents is a dictionary.
- paste (instance derived from ActionBase) – paste action
- static (boolean) – flag indicating whether the specification contains dynamic elements
This action inserts the given contents into the system clipboard, and then performs the paste action to paste it into the foreground application. By default, the paste action is the Ctrl-v keystroke or Super-v on a mac. The default clipboard format used by this action is the Unicode text format.
Mouse action¶
This section describes the Mouse
action object. This type of
action is used for controlling the mouse cursor and clicking mouse
button.
Below you’ll find some simple examples of Mouse
usage, followed
by a detailed description of the available mouse events.
Example mouse actions¶
The following code moves the mouse cursor to the center of the foreground
window ((0.5, 0.5)
) and then clicks the left mouse button once
(left
):
# Parentheses ("(...)") give foreground-window-relative locations.
# Fractional locations ("0.5", "0.9") denote a location relative to
# the window or desktop, where "0.0, 0.0" is the top-left corner
# and "1.0, 1.0" is the bottom-right corner.
action = Mouse("(0.5, 0.5), left")
action.execute()
The line below moves the mouse cursor to 100 pixels left of the primary
monitor’s left edge (if possible) and 250 pixels down from its top edge
([-100, 250]
), and then double clicks the right mouse button
(right:2
):
# Square brackets ("[...]") give desktop-relative locations.
# Integer locations ("1", "100", etc.) denote numbers of pixels.
# Negative numbers ("-100") are counted from the left-edge of the
# primary monitor. They are used to access monitors above or to the
# left of the primary monitor.
Mouse("[-100, 250], right:2").execute()
The following command drags the mouse from the top right corner of the
foreground window ((0.9, 10), left:down
) to the bottom left corner
((25, -0.1), left:up
):
Mouse("(0.9, 10), left:down, (25, -0.1), left:up").execute()
The code below moves the mouse cursor 25 pixels right and 25 pixels up
(<25, -25>
):
# Angle brackets ("<...>") move the cursor from its current position
# by the given number of pixels.
Mouse("<25, -25>").execute()
Mouse specification format¶
The spec argument passed to the Mouse
constructor specifies
which mouse events will be emulated. It is a string consisting of one or
more comma-separated elements. Each of these elements has one of the
following possible formats:
Mouse movement actions:
- move the cursor relative to the top-left corner of the desktop monitor containing coordinates
[0, 0]
(i.e. the primary monitor):[
number,
number]
- move the cursor relative to the foreground window:
(
number,
number)
- move the cursor relative to its current position:
<
pixels,
pixels>
In the above specifications, the number and pixels have the following meanings:
number – can specify a number of pixels or a fraction of the reference window or desktop. For example:
(10, 10)
– 10 pixels to the right and down from the foreground window’s left-top corner(0.5, 0.5)
– center of the foreground windowpixels – specifies the number of pixels
- Mouse button-press action:
- keyname [
:
repeat] [/
pause]keyname – Specifies which mouse button to click:
left
– left mouse button keymiddle
– middle mouse button keyright
– right mouse button keyfour
– fourth mouse button keyfive
– fifth mouse button keywheelup
– mouse wheel upstepup
– mouse wheel up 1/3wheeldown
– mouse wheel downstepdown
– mouse wheel down 1/3wheelright
– mouse wheel rightstepright
– mouse wheel right 1/3wheelleft
– mouse wheel leftstepleft
– mouse wheel left 1/3
repeat – Specifies how many times the button should be clicked:
0
– don’t click the button, this is a no-op1
– normal button click2
– double-click3
– triple-click
pause – Specifies how long to pause after clicking the button. The value should be an integer giving in hundredths of a second. For example,
/100
would mean one second, and/50
half a second.
- Mouse button-hold or button-release action:
- keyname
:
hold-or-release [/
pause]keyname – Specifies which mouse button to click; same as above.
hold-or-release – Specified whether the button will be held down or released:
down
– hold the button downup
– release the button
pause – Specifies how long to pause after clicking the button; same as above.
Mouse across platforms¶
Please note that there are some platforms which do not support emulating
every mouse button listed above. If an unsupported mouse button (keyname)
is specified and the Mouse
action executed, an error is raised. For
instance, scrolling the mouse wheel horizontally (e.g. wheelleft) is not,
by default, a supported operation on X11:
ValueError: Unsupported scroll event: wheelleft
Fortunately, this particular problem can be fixed by installing the pynput library:
pip install pynput
On MacOS, however, Dragonfly cannot be used to scroll horizontally.
Function action¶
The Function
action wraps a callable, optionally with some
default keyword argument values. On execution, the execution data
(commonly containing the recognition extras) are combined with the
default argument values (if present) to form the arguments with which
the callable will be called.
Simple usage:
>>> def func(count):
... print("count: %d" % count)
...
>>> action = Function(func)
>>> action.execute({"count": 2})
count: 2
True
>>> # Additional keyword arguments are ignored:
>>> action.execute({"count": 2, "flavor": "vanilla"})
count: 2
True
Usage with default arguments:
>>> def func(count, flavor):
... print("count: %d" % count)
... print("flavor: %s" % flavor)
...
>>> # The Function object can be given default argument values:
>>> action = Function(func, flavor="spearmint")
>>> action.execute({"count": 2})
count: 2
flavor: spearmint
True
>>> # Arguments given at the execution-time to override default values:
>>> action.execute({"count": 2, "flavor": "vanilla"})
count: 2
flavor: vanilla
True
Usage with the remap_data
argument:
>>> def func(x, y, z):
... print("x: %d" % x)
... print("y: %d" % y)
... print("z: %d" % z)
...
>>> # The Function object can optionally be given a second dictionary
>>> # argument to use extras with different names. It should be
>>> # compatible with the 'defaults' parameter:
>>> action = Function(func, dict(n="x", m="y"), z=4)
>>> action.execute({"n": 2, "m": 3})
x: 2
y: 3
z: 4
True
Class reference¶
-
class
Function
(function, remap_data=None, **defaults)[source]¶ Call a function with extra keyword arguments.
- Constructor arguments:
- function (callable) – the function to call when this action is executed
- remap_data (dict, default: None) – optional dict of data keys to function keyword arguments
- defaults – default keyword-values for the arguments with which the function will be called
Mimic action¶
The Mimic
action mimics a single recognition. This is useful for
repeating a single prerecorded or predefined voice-command.
This class could for example be used to open a new Windows Explorer window:
action = Mimic("open", "windows", "explorer")
action.execute()
A more in-depth example is given below in the class reference.
Mimic quirks¶
Some SR engine back-ends have confusing engine.mimic()
method
behavior. See the engine-specific mimic method documentation in sections
under Engines sub-package for more information.
Class reference¶
-
class
Mimic
(*words, **kwargs)[source]¶ Mimic recognition action.
The constructor arguments are the words which will be mimicked. These should be passed as a variable argument list. For example:
action = Mimic("hello", "world", r"!\exclamation-mark") action.execute()
If an error occurs during mimicking the given recognition, then an ActionError is raised. A common error is that the engine does not know the given words and can therefore not recognize them. For example, the following attempts to mimic recognition of one single word including a space and an exclamation-mark; this will almost certainly fail:
Mimic("hello world!").execute() # Will raise ActionError.
The constructor accepts the optional extra keyword argument, and uses this to retrieve dynamic data from the extras associated with the recognition. For example, this can be used as follows to implement dynamic mimicking:
class ExampleRule(MappingRule): mapping = { "mimic recognition <text> [<n> times]": Mimic(extra="text") * Repeat(extra="n"), } extras = [ IntegerRef("n", 1, 10), Dictation("text"), ] defaults = { "n": 1, }
The example above will allow the user to speak “mimic recognition hello world! 3 times”, which would result in the exact same output as if the user had spoken “hello world!” three times in a row.
Playback action¶
The Playback
action mimics a sequence of recognitions. This is
for example useful for repeating a series of prerecorded or predefined
voice-commands.
This class could for example be used to reload with one single action:
action = Playback([
(["focus", "Natlink"], 1.0),
(["File"], 0.5),
(["Reload"], 0.0),
])
action.execute()
Mimic quirks¶
Some SR engine back-ends have confusing engine.mimic()
method
behavior. See the engine-specific mimic method documentation in sections
under Engines sub-package for more information.
Class reference¶
-
class
Playback
(series, speed=1)[source]¶ Playback a series of recognitions.
- Constructor arguments:
- series (sequence of 2-tuples) – the recognitions to playback. Each element must be a 2-tuple of the form ([“words”, “two”, “mimic”], interval), where interval is a float giving the number of seconds to pause after the given words are mimicked.
- speed (float) – the factor by which to speed up playback. The intervals after each mimic are divided by this number.
-
speed
¶ Factor to speed up playback.
WaitWindow action¶
-
class
WaitWindow
(title=None, executable=None, timeout=15)[source]¶ Wait for a specific window context action.
- Constructor arguments:
- title (str) – part of the window title: not case sensitive
- executable (str) – part of the file name of the executable; not case sensitive
- timeout (int or float) –
the maximum number of seconds to wait for the correct
context, after which an
ActionError
will be raised.
When this action is executed, it waits until the correct window context is present. This window context is specified by the desired window title of the foreground window and/or the executable name of the foreground application. These are specified using the constructor arguments listed above. The substring search used is not case sensitive.
If the correct window context is not found within timeout seconds, then this action will raise an
ActionError
to indicate the timeout.
FocusWindow action¶
-
class
FocusWindow
(executable=None, title=None, index=None, filter_func=None, focus_only=False)[source]¶ Bring a window to the foreground action.
- Constructor arguments:
- executable (str) – part of the filename of the application’s executable to which the target window belongs; not case sensitive.
- title (str) – part of the title of the target window; not case sensitive.
- index (str or int) – zero-based index of the target window, for multiple matching windows; can be a string (for substitution) but must be convertible to an integer.
- filter_func (callable) – called with a single argument
(the window object), and should return
True
for your target windows; example:lambda window: window.get_position().dy > 100
. - focus_only (bool, default False) – if True, then attempt to focus the window without raising it by using the Window.set_focus() method instead of set_foreground(). This argument may do nothing depending on the platform.
This action searches all visible windows for a window which matches the given parameters.
BringApp and StartApp actions¶
The StartApp
and BringApp
action classes are used to
start an application and bring it to the foreground. StartApp
starts an application by running an executable file, while
BringApp
first checks whether the application is already running
and if so brings it to the foreground, otherwise starts it by running the
executable file.
Example usage¶
The following example brings Notepad to the foreground if it is already open, otherwise it starts Notepad:
BringApp(r"C:\Windows\system32\notepad.exe").execute()
Note that the path to notepad.exe given above might not be correct for your computer, since it depends on the operating system and its configuration.
In some cases an application might be accessible simply through the file name of its executable, without specifying the directory. This depends on the operating system’s path configuration. For example, on the author’s computer the following command successfully starts Notepad:
BringApp("notepad").execute()
Applications on MacOS are started and switched to using the application name:
BringApp("System Preferences").execute()
Class reference¶
-
class
BringApp
(*args, **kwargs)[source]¶ Bring an application to the foreground, starting it if it is not yet running.
When this action is executed, it looks for an existing window of the application specified in the constructor arguments. If an existing window is found, that window is brought to the foreground. On the other hand, if no window is found the application is started.
Note that the constructor arguments are identical to those used by the
StartApp
action class.- Constructor arguments:
- args (variable argument list of str’s) –
these strings are passed to
subprocess.Popen()
to start the application as a child process - cwd (str, default None) – if not None, then start the application in this directory
- title (str, default None) – if not None, then match existing windows using this title.
- index (str or int) – zero-based index of the target window, for multiple matching windows; can be a string (for substitution) but must be convertible to an integer.
- filter_func (callable) – called with a single argument
(the window object), and should return
True
for your target windows; example:lambda window: window.get_position().dy > 100
. - focus_after_start (bool, default False) – if True, then attempt to bring the window to the foreground after starting the application. Does nothing if the application is already running.
- focus_only (bool, default False) – if True, then attempt to focus a matching window without raising it by using the set_focus() method instead of set_foreground(). This argument may do nothing depending on the platform.
- args (variable argument list of str’s) –
these strings are passed to
-
class
StartApp
(*args, **kwargs)[source]¶ Start an application.
When this action is executed, it runs a file (executable), optionally with commandline arguments.
- Constructor arguments:
- args (variable argument list of str’s) – these strings are passed to subprocess.Popen() to start the application as a child process
- cwd (str, default None) – if not None, then start the application in this directory
- focus_after_start (bool, default False) – if True, then attempt to bring the window to the foreground after starting the application.
A single list or tuple argument can be used instead of variable arguments.
RunCommand action¶
The RunCommand
action takes a command-line program to run including
any required arguments. On execution, the program will be started as a
subprocess.
Processing will occur asynchronously by default. Commands running asynchronously should not normally prevent the Python process from exiting.
It may sometimes be necessary to use a list for the action’s command
argument instead of a string. This is because some command-line shells may
not work 100% correctly with Python’s built-in shlex.split()
function.
This action should work on Windows and other platforms.
Example using the ping command:
from dragonfly import RunCommand
# Ping localhost for 4 seconds.
RunCommand('ping -w 4 localhost').execute()
Example using a command list instead of a string:
from dragonfly import RunCommand
# Ping localhost for 4 seconds.
RunCommand(['ping', '-w', '4', 'localhost']).execute()
Example using the optional function parameter:
from __future__ import print_function
from locale import getpreferredencoding
from six import binary_type
from dragonfly import RunCommand
def func(proc):
# Read lines from the process.
encoding = getpreferredencoding()
for line in iter(proc.stdout.readline, b''):
if isinstance(line, binary_type):
line = line.decode(encoding)
print(line, end='')
RunCommand('ping -w 4 localhost', func).execute()
Example using the optional synchronous parameter:
from dragonfly import RunCommand
RunCommand('ping -w 4 localhost', synchronous=True).execute()
Example using the optional hide_window parameter:
from dragonfly import RunCommand
# Use hide_window=False for running GUI applications via RunCommand.
RunCommand('notepad.exe', hide_window=False).execute()
Example using the subprocess’s Popen
object:
from dragonfly import RunCommand
# Initialise and execute a command asynchronously.
cmd = RunCommand('ping -w 4 localhost')
cmd.execute()
# Wait until the subprocess finishes.
cmd.process.wait()
Example using a subclass:
from __future__ import print_function
from locale import getpreferredencoding
from six import binary_type
from dragonfly import RunCommand
class Ping(RunCommand):
command = "ping -w 4 localhost"
synchronous = True
def process_command(self, proc):
# Read lines from the process.
encoding = getpreferredencoding()
for line in iter(proc.stdout.readline, b''):
if isinstance(line, binary_type):
line = line.decode(encoding)
print(line, end='')
Ping().execute()
Class reference¶
-
class
RunCommand
(command=None, process_command=None, synchronous=False, hide_window=True)[source]¶ Start an application from the command-line.
This class is similar to the
StartApp
class, but is designed for running command-line applications and optionally processing subprocesses.- Constructor arguments:
- command (str or list) – the command to run when this
action is executed. It will be parsed by
shlex.split()
if it is a string and passed directly tosubprocess.Popen
if it is a list. Command arguments can be included. - process_command (callable) – optional callable to invoke
with the
Popen
object after successfully starting the subprocess. Using this argument overrides theprocess_command()
method. - synchronous (bool, default False) – whether to wait
until
process_command()
has finished executing before continuing. - hide_window (bool, default True) – whether to hide the application window. Set to False if using this action with GUI programs. This argument only applies to Windows. It has no effect on other platforms.
- command (str or list) – the command to run when this
action is executed. It will be parsed by
-
process
¶ The
Popen
object for the current subprocess if one has been started, otherwiseNone
.
ContextAction¶
-
class
ContextAction
(default=None, actions=None)[source]¶ Action class to execute a different action depending on which context is currently active.
This is especially useful for allowing the same commands to work in multiple applications without redefining them in other grammars. An example of this is the redo shortcut. Some applications use Ctrl+Shift+Z, while others might use Ctrl+Y instead.
ContextAction
could be used to define Ctrl+Y as the default and use Ctrl+Shift+Z for specific contexts:redo = ContextAction(default=Key('c-y'), actions=[ # Use cs-z for rstudio (AppContext(executable="rstudio"), Key('cs-z')), ])
This class was originally written for the Aenea project by Alex Roper and has been modified to work without Aenea’s functionality.
- Constructor arguments:
- default (action object, default do nothing) – the default action to execute if there was no matching context in actions.
- actions (iterable, default empty list) – an iterable object containing context-action pairs. The action of the first matching context will be executed.
-
add_context
(context, action)[source]¶ Add a context-action pair to the actions list.
Parameters: - context (Context) – dragonfly context
- action (ActionBase) – dragonfly action
Pause action¶
-
class
Pause
(spec=None, static=False)[source]¶ Pause for the given amount of time.
The spec constructor argument should be a string giving the time to wait. It should be given in hundredths of a second. For example, the following code will pause for 20/100s = 0.2 seconds:
Pause("20").execute()
The reason the spec must be given as a string is because it can then be used in dynamic value evaluation. For example, the following code determines the time to pause at execution time:
action = Pause("%(time)d") data = {"time": 37} action.execute(data)
PlaySound action¶
The PlaySound
action class is used to play wave files.
Example usage¶
The following example shows how to play a wave file using the
PlaySound
action class:
PlaySound(file="tada.wav").execute()
Windows¶
On Windows, PlaySound
uses the PlaySound Windows API function.
The action can be used to play Windows system sounds. For example:
# Play the system shutdown sound.
PlaySound("SystemExit").execute()
# Play the logout sound.
PlaySound("WindowsLogout").execute()
System sound names are matched against registry keys.
Invalid file paths or unknown system sounds will result in the default error
sound being played. RuntimeErrors
will be raised if Windows fails to
play a known system sound.
Other platforms¶
On other platforms, the PlaySound
class will use PyAudio to play specified wave
files.
Invalid file paths will result in errors on other platforms.
Class reference¶
-
class
PlaySound
(name='', file=None)[source]¶ Start playing a wave file or system sound.
When this action is executed, the specified wave file or named system sound is played.
Playing named system sounds is only supported on Windows.
- Constructor arguments:
- name (str, default empty string) – name of the Windows system sound to play. This argument is effectively an alias for file on other platforms.
- file (str, default None) – path of wave file to play when the action is executed.
If name and file are both None, then waveform playback will be silenced on Windows when the action is executed. Nothing will happen on other platforms.