I’m going to let you in on a secret. Don’t tell anyone though.

While working (Software Developer/Tech Writing), I haven’t used my keyboard for the entire week for work related tasks. Not only that, I’ve been more productive than otherwise.

I’ve also not used my keyboard for REAPER since late December. Soon, I won’t be using my mouse at all either.

I’m using Talon Voice and it’s been amazing. Let me walk you through how this works with REAPER, and I’ll explain my Dev use in another post.

Video
Preface
Talon
Hasn’t this all been a setup!?
OSC
Commands
- How I got started
Additions
REAPER’s Midi Editor
The Code
Conclusion
Addendum
Meta

Video

Preface

Before you read this, note that some things have changed before I finalized this post. Some code/text might not match the repository.

More importantly, I’m not a Python dev and this is a somewhat opinionated Python scripting environment. Many things that you might see and think “What, you could do it like…”, you can’t. Most of the time it’ll just be my fault :)

Talon

The application that I’m using for this is called Talon. It allows voice control, noise control, and (single monitor) eye tracking.

I use only the voice control portion currently. I tried eye tracking and learned that I stare off into space while I use my computer too often (which is probably why I use my mouse like this!)

Talon has a simple command syntax that maps word recognition to some sort of keyboard output, or triggers a python script (which can do basically anything on your computer). Python scripts themselves can access word recognition and do what they please.

Dragon supports a number of engines for audio recognition (not just speech, but some noises as well). Dragon is still the indisputed king in this realm and Talon can utilize Dragon, but Dragon is dead on macOS. I’ve had a great deal of maintaining a working Dragon, so I instead use a wav2letter model trained by the Talon author(/team?). The wav2letter model available to Talon Beta users works incredibly well.

If this interests you then please Join the Patreon and Join the Slack Channel.

Hasn’t this all been a setup!?

You might think that I spent all this time working through key commands in REAPER to prepare for this transition. I have not. I started using Talon on December 13th. This series was started on November 9th.

It just happens that my views on interactive design align nearly perfectly with voice control.

I was able to take a list of REAPER commands, apply a regex replace and immediately begin to work.

OSC

I use key commands for most things in REAPER, but I use OSC for various commands. So this means setting up OSC in REAPER.

In Preferences->Control/OSC/Web I add a new OSC “device” with:

Mode - Configure device IP+local port
Local listen port - 8000
Local IP - default value, or 127.0.0.1
Device port - 9000
Device IP - default value, or 127.0.0.1
Allow binding messages to REAPER actions and FX Learn - checked

The rest is default.

Commands

How I got started

To create the initial Talon command file, I took this post, copied the action segments and ran the following regex -> replacement

^#+.*\n -> ``
- Meaning - Find one or more of # at the start of the line followed by any number of any characters followed by a new line. Replace with nothing.
  - Remove all headers
✓ -> ``
- Meaning - Replace all checkmarks with nothing.
^ .*\n - ``
- Meaning - Find all occurrences of two spaces at the beginning of the line that are followed by any characters then a new line. Replace with nothing.
  - Removes secondary explanations that aren’t commands.
+ -> ``
- Meaning - Find all occurrences of one space followed by one or more spaces. Replace with nothing.
  - Remove all double spaces to make further processing easier. Important that this happens after the last step.

Then replace control with ctrl, comand with cmd and option with alt.

At this point I had a bunch of lines that looked like this: * **h** View: Move cursor left to grid division``

To turn these into Talon commands I used:

\* \*\*(.*?)\*\* -? ?`(.*?:|) ?(.*?)`.*\n

Replace with:

\3:\n    key(\1)

Meaning - Find literal * ** and capture text up till the next literal ** as group 1. Match a space followed by 0 or more - then a backtick. Match all text up to a : or nothing as group 2. Match all text up to the next backtick followed by a new line as capture group 3. Replace with capture group 3, colon, new line, 4 spaces, then capture group 1 inside the parenthesis of key().

From there I needed to hand edit some commands (remove superfluous phrases like “to cursor” or “move to”), but I had many things like:

Zoom time selection:
    key(z)

Now if I say “Zoom time selection”, then the key z is pressed.

In about 5 minutes I was able to take everything I’ve written about and turn it into a voice command.

Additions

Anything that adds a new action will be added to day 23’s list of commands.

Find Track

I want to be able to say “Find Track Stereo Kick”. So this means creating a capture in Talon.

I add the following to a file named reaper.talon

find track <phrase>$:
     key("alt-f2")
     insert(phrase)
     key("enter")

This command “find track” that also captures the words that follow that phrase. I then use my focus track manager script, enter the text from the phrase and hit enter.

Now the tracks matching that phrase, and their children, are displayed.

Go to Track

Often I want to go to a numbered track, since I always have a view of the number on the left hand side of the track control panel, or the mixer control panel.

I use the python package pythonosc to send OSC messages to REAPER.

# Requires ~/.talon/bin/pip install python-osc
from pythonosc import udp_client

client =  udp_client.SimpleUDPClient("127.0.0.1", 8000)

# REAPER must be set up to allow OSC control of tracks.
client.send_message("/reaper/track/follows", "DEVICE")
client.send_message("/device/track/follows", "LAST_TOUCHED")

@mod.capture(rule="go to track <number>")
def go_to_track(m) -> str:
    client.send_message("/device/track/select", m.number)
    return "" 

This is pretty simple, I just send a OSC message directly to REAPER “go to track” capture.

I use a similar capture for:

Setup for OSC use
- “Initialize REAPER” - Sets up reaper for accepting certain OSC commands.
Record enabling tracks
- “Record Arm 6” - Record enables track 6
- “Record Disarm 6” - Record disables track 6
Current track name
- “Track name nerd” - Current track name changed to “nerd”
Track name by track number
- “Track 10 name super nerd” - Set track 10’s name to “super nerd”
Track Volume
- “Track Volume 6” - Set track to +6dB
- “Track Volume minus 5” - Set track to -5dB
- “Track 20 Volume -2” - Set track 20’s volume to -2dB
- “Master Volume -4” - Set master track volume to -4dB
Track Pan
- “Track Pan 10 left” - Sets track pan to 10 left
- “Track pan 76 percent” - Sets track pan to 51%
- “Track 2 pan 10 percent” - Sets track 2’s pan to 10%
- “Master Pan 51 percent” - Set the master track volume to 1% right.
Track Send Volume
- “Send 1 Volume -10” - Sets the volume of send 1 to -10dB
- “Master Send 3 Volume 3” - Set the volume of the master track’s 3rd send to 3dB
Track Send Pan
- “Send 3 Pan 75%” - Sets the pan of send 3 to 75%
- “Master Send 3 pan 2 left” - Set the pan of the master track’s 2nd send to 2 left
Open Effect UI
- “Open Effect 2” - Opens Effect #2 on the track
Bypass/UnBypass Effects on the current track (Using bypass/unbypass or enable/disable)
- “Enable Effect 3” - Unbypass Effect 3
- “Bypass Effect 1” - Bypass Effect 1

Note: Since doing this, I’ve realized that nearly every OSC command I’ve setup has a better alternative, but they’re all over the place! I still like the idea of a central communication method. Possibly the Web control in the future?

Update State

Soon you’ll be wondering how you get REAPER to barf up its current state.

The action Control surface: Refresh all surfaces does this.

client.send_message("/action", 41743) is the code.

REAPER will spit up a lot of information, so be careful when you use this! This can be reduced by editing Reaper.DefaultOSC to remove/comment the data that you don’t need to see.

I only use it when showing a GUI and on initialize.

FX Parameters

FX Parameters means getting data from REAPER. To do this I run an OSC server and keep a dict of the most recently sent 1024 parameters.

There’s only a need for parameter information for the currently shown effect, so there’s no need to track anything but the latest data that REAPER sends… yet.

A simple server example (note that Talon does not yet support asyncio):

import atexit
import threading
from typing import List, Any
from pythonosc import dispatcher, osc_server

def fxparam_updater(address: str, *args: List[Any]):
    print(f"Number: {fx_int}, Key: {fx_key}, val: {args[0]}")


def server_thread():
    d = dispatcher.Dispatcher()
    d.map("/fxparam/*", fxparam_updater)
    server = osc_server.ThreadingOSCUDPServer(("", 9000), d)
    server.allow_reuse_address = True
    server.serve_forever()

x = threading.Thread(target=server_thread)
x.start()

@atexit.register
def shutdown_server():
    x.exit()

To install python-osc, it must be done with Talon’s pip. This is located in ~/.talon, so you need to do something like ~/.talon/bin/pip install python-osc.

From there I create a GUI with talon that allows me to cycle between banks of the 1024 effects parameters.

import user.robert_talon.reaperserver as rs

@imgui.open(y=0, software=app.platform == "linux")
def param_gui(gui: imgui.GUI):
    gui.text(f"Current Bank: {rs.cur_display_bank}")

    for i in range(rs.fx_display_size):
        offset = (rs.cur_display_bank - 1) * rs.fx_display_size
        cur_index = i + offset
        if cur_index > rs.fx_bank_size:
            break;
        gui.line()
        d = rs.cur_params[cur_index]
        gui.text("{}: {} -- {}".format(cur_index + 1, d['name'], d['value']))

That relies on code additional to the server example (currently viewed effect state stored as globals, as complexity warrants this will be made less gross. Perhaps more accurately - when I better understand concurrency in Python, I’ll change it.)

Then I can use a command like “Parameter 3 update 40 percent” to update parameter 3 to 40 percent.

Parameter Banks

It’s no good if a plugin supports 100s of parameters and I can only see what fits on my screen.

I tell reaper that I support an FX parameter bank size of 1024 client.send_message("/device/fxparam/count", 1024) and then maintain a dict with those settings.

Separately I keep a display size, and current bank. The function that displays the parameters uses the display size, and current bank as an offset, to display the parameters in the current viewing bank.

I can say things like “Parameter bank up”, “Parameter bank down”, and more importantly, “Parameter bank size 64” to change what I’m currently viewing of the current FX’s parameters.

The parameter’s values are updated in realtime as well.

Parameter Saving

There’s a lot of parameters for some effects/instruments!

Wouldn’t it be nice if you could say “show parameters” and maybe “parameter bank up” or “parameter bank down” a few times, then say “Save parameter 4”?

Cool. I can do that.

Favorites

Whew! Some plugins have 1000+ parameters. I don’t touch but maybe a dozen or two of them for any plugin, and browsing for the right parameter (or remembering it) each time is an absurd expectation.

I can say “save parameter 5” and a dict of {plugin_name: {index: parameter_name}} is updated.

Any time I pull up an effect with the same name I can say “Favorite parameters” and it will show me a list of the parameter number, name and its current value!

These favorites persist to disk.

Parameter changing

Need to change a parameter? Ok. “Parameter 5 update 20 percent”.

All parameter changes are in percentage currently. There’s some complexity here that I need to explore.

Markers

“Show markers” and “go to marker 2” work as you might now expect. Same for Regions. (replace ‘markers’ with ‘regions’)

“Marker 4 name First Verse” will name marker 4 to “First Verse”. Same for Regions. (replace ‘marker’ with ‘region’)

“Edit Marker” will bring up the dialog to edit a marker. Same for Regions. (replace ‘marker’ with ‘region’)

I do not have functionality for editing markers/regions, as I’ve found it easier to delete them and create another. For my uses, retaining the ID’s associated with a marker/region is not valuable.

Arbitrary Actions

I can send arbitrary actions using the COMMAND ID using these functions:

def send_osc_action(n: int):
    client.send_message("/action", n)

def send_MIDI_osc_action(n: int):
    client.send_message("/midiaction", n)

Then in my .talon file something like:

create multiple tracks:
    user.send_osc_action(41067)

This is useful for when I want to triggering action, but I do not want to assign it to a war shortcut.

Reset/Setup

REAPER needs to be setup for OSC input to control tracks by editing Default.ReaperOSC in your settings folder, or using code like this:

@mod.capture(rule="(initialize|reset|setup) reaper")
def initialize_reaper(m) -> str:
    client.send_message("/reaper/track/follows", "DEVICE")
    client.send_message("/device/track/follows", "LAST_TOUCHED")
    return "" 

If setting these via OSC, it must be done after REAPER is opened, or various /track commands will not work correctly via OSC.

I have my Talon setup to send these messages on load, but often REAPER is loaded after Talon.

I can say “initialize reaper” or “reset reaper” or “setup reaper” to ensure that my track navigation commands work. I could also edit these settings in the Default.ReaperOSC file, but that’s not as portable of an option.

Input Assignment

REAPER has no way to set input via OSC or command, but we can use the amazing ReaConsole to do this!

I have an action to send OSC messages at will in my reaper.py:

@mod.action_class
class Actions:
    def send_osc_msg(s: str, n: Any):
        """sends an OSC message to create an action"""
        client.send_message(s, n)

    def send_osc_action(n: int):
        """sends an OSC message to create an action"""
        client.send_message("/action", n)

    def send_osc_action_str(s: str):
        """sends an OSC message to create an action"""
        client.send_message("/action", s)

    def send_osc_toggle_action(n: int):
        """sends an OSC message to create an action"""
        client.send_message(f"/action/{n}", 1)

Then in my reaper.talon I can trigger ReaConsole’s SWS: Open console with 'i' to set track(s) input and enter the input assignment.

<user.input_assign>:
    user.send_osc_action_str("_SWSCONSOLEINPUT")
    sleep(200ms)
    insert(input_assign)
    key("enter")
    key(esc)

Do a thing to many things

I use a “do thing through " paradigm so much that I defined this action for use with ReaConsole:

@mod.action_class
class Actions:
    def console_through(s: str, n: int, n2: int, open: int = 0):
        """ReaConsole command for doing things to multiple tracks"""
        client.send_message("/action", s)
        big_n, little_n = (n2, n) if n2 > n else (n, n2)
        actions.sleep("200ms")
        actions.insert(little_n)
        actions.insert("-")
        actions.insert(big_n)
        actions.insert(" ")
        if not open:
            actions.key("enter")
            actions.key("esc")

Then in my reaper.talon I can trigger ReaConsole’s SWS: Open console with 'i' to set track(s) input and enter the input assignment.

select tracks <number> through <number>:
    user.console_through("_SWSCONSOLEEXSEL", number_1, number_2)

I can say “Select tracks 36 through 114” and it will select those tracks inclusively. Or “Mute tracks 1 through 5” or “Solo tracks 10 through 2” (larger number first).

(Note Talon Users: It took me a while to figure out how to refer to multiple numbers in a command. Take note of number_1 and number_2 there.)

Alternatively I can leave the console open for something like adding a track name prefix:

prefix tracks <number> through <number>:
    user.console_through("_SWSCONSOLEPREFIX", number_1, number_2, 1)

Now I can say “Prefix tracks 3 through 10 verse underscore enter escape” to prefix “verse_” to tracks 3 through 10.

Select Item

NEW ACTION

cmd-option-enter - Xenakios/SWS: Select items under edit cursor on selected tracks

I have cursor movement down fairly well, and track selection down. Now I need a way to select an item. Done.

And More…

I’ll add all the available commands to my REAPER settings after this series is finished.

REAPER’s Midi Editor

Sucks.

I spent way too much time dealing with missing shortcuts, laggy behaviour (which affects fast input from Talon), names for ortherwise identical actions with different names from the Main context, focus issues, extremely poor step editor and…

Ugh.

Not a pleasant experience.

The Code

My REAPER setup is constantly evolving, but you can clone, watch or issue pull requests to my Talon setup.

Conclusion

I can achieve all basic (that I’ve needed!) recording/editing tasks with REAPER by voice.

This is potentially useful for anyone with disabilities (like myself), injuries or someone that wishes to control their REAPER software remotely, such as with a wireless lapel mic as they move around the studio.

This entire post has also been written using talon. With my emacs and vim setup, I have been able to efficiently write and edit this entire document. Many things have been more difficult than typing, but many things have been more efficient as well. Based on my time tracking, the net result has been more efficiency by a small margin.

If there is sufficient interest then I will cover my programming and writing set up as well. This is all changing as I get used to it and learn what I need from the system, but maybe my experiments will help you get started.

Addendum

I’ve done a good bit of experimenting with REAPER’s OSC, and I’ve since decided that it’s an awful option. The web interface is a better option, but I didn’t want to spend another 2 weeks yakshaving before getting out a proof of concept.

Reaper Day 24 - Accessibility - Look Ma! No Hands

Contents