Al powered voice controls + mnemonics = magic.
I’m going to be reworking this article over the next few weeks. Think of it is as a work-in-progress that sketches out one of my visions for the future, namely a world where we don’t have to stare into computer monitors all the time.
Recently, Farza shared a clip of himself controlling his computer using only his voice with HeyClicky:
Mnemonics is an oral technology. Pretty much any sufficiently developed oral culture will have some sort of mnemonic tradition. Lynn Kelly’s Memory Code is an excellent primer on the mnemonic practices of different cultures, it’s well worth the read.
Of course, the printing press did a number on the art of memory, and the art has fallen into steep decline.
AI, though, is the best thing that’s ever happened to mnemonics.
It gives it a purpose again.
I do not like using my computer. I dream of the day where I only have to spend a few hours a month looking into a screen.
Voice control technologies might allow us to do this. Of course, HeyClicky is being used in conjunction with a computer screen, but with mnemonics, we can cut down on our need for screens in a variety of contexts.
Using AI and mnemonics, it’s possible to engage in dictation, composition, and other creative acts entirely by voice, no visual feedback necessary.
And I’m not talking about asking the AI to “write for you”, but about using the AI to manage where your writing goes, how it’s published, and where to send it.
The goal is to use AI to manage contexts, not content. Humans should still create, but the robots can help us create in a way that minimizes our exposure to the digital universe.
Mnemonics, meanwhile, allows us to manage workflows and AI behaviour without the need for visual references.
This newsletter is part of a larger project of mine that aims to minimize our reliance on digital screens. That might involve using volumetric displays for Zoom calls, e-ink screens for writing and illustrations, or even connecting an abacus to AI for displaying calculations.
That might seem like a silly example, but it helps set the stage for what I’m talking about. I’m mostly agnostic as to what we replace our computer screens with, so long as they get replaced.
Voice controlled interfaces hold the most potential, but they’ll often be used in conjunction with other tools.
AI isn’t going away. What we need to do is develop a healthy relationship with the technology. One of the healthiest applications of the technology is minimizing the time we spend on our computers.
Mnemonics, when used with AI powered voice interfaces, can allow us to spend less time online.
A W.I.P Article Road Map
Current Dictation Software
1.1 Native solutions
1.2 API solutions
Voice Controlled Interfaces
2.1 Screen Aware Copilots like HeyClicky,
2.2 Full OS Voice Tools like Talon
2.3 AI Assistants
2.4 Voice-to-text tools
My Experimental Mnemonic Interfaces
Mnemonics For Dictation
4.1 Thomas Aquinas
4.2 Miraculous Dictation
Using Mnemonics with AI
5.1 Question Tables inspired by the Lullian Alphabet
5.2 The Ars Combinatoria
5.3 Other Investigatory Techniques
Novel Analog Displays
Managing Physical Robots With Mnemonics
The Limits of Mnemonic Interfaces
First & Third Party Implementations
Withered Technologies and Lateral Design
There’s more than just the ten sections above, but I’ll keep it limited to that for now. I’m releasing this W.I.P because otherwise I’ll just keep putting this off.
May as well jump into the deep-end from the start.
Current Dictation Software
Native solutions
All major operating systems come with native dictation software.
If you have a Mac, you can enable dictation by going to your system settings keyboard menu, and enabling dictation. You can set a hot key, or use the F5 default, or turn dictation on via the edit menu of whatever text editor you’re using.
On Windows, press the Windows logo key + H in an active text field and begining speaking when the voice typing toolbar appears.
Dictation is like a stylus for your mouth. It’s text entry using words. It’s not voice control, so you won’t be able to interact with your computer’s user interface, but it’s still very useful for acts of composition.
If you paid dictation software with some of the mnemonic techniques I’ve shared in this newsletter, you can do quite a lot with it.
You can easily compose articles on your computer without having to look at your monitor using these dictation tools.
Dictation software is a good first layer because it helps you establish the habit of speaking into a machine instead of typing on one.
You want to build this muscle up.
API Dictation
API-based dictation is where things start getting spicy.
Instead of simply placing text to a field, you can start building custom voice systems with domain-specific workflows. Native dictation just takes what you give and puts it into text, but API-based dictation can transform what you say, in real time.
It can classify intent, clean up prose, expand shorthand, summarize arguments, or route your speech into structured formats.
It makes dictation programmable. A spoken phrase can become an instruction, a database entry, a note, a command, or a draft paragraph — all without switching windows or opening new applications.
You can use the API to create a voice interface layer that will know how to respond to different voice commands or prompts.
You’re no longer locked into a single text window.
Call Bob. E-mail Dobbs. Buy a pizza. Load the soundtrack to Empire Records. Play The Peanut Butter Solution on Screen 1 and Goonies on Screen 2.
Each of those voice commands could then be routed to the appropriate execution layer.
Using dictation APIs and a bit of programming know-how, you can turn items in your memory palace into computer actions.
Voice Controlled Interfaces
2.1 Screen-aware copilots like HeyClicky
With API-based dictation, you have to program your own solutions, whereas screen-aware copilots like HeyClicky work out of the box.
Copilots will be less versatile than the personal solutions that API-based dictation enables, but they’re going to be much easier to setup. This is how most people are going to get started with mnemonic interfaces.
Using mnemonics, you can set up workflows for your copilot that would be impossible to do if you were only relying on working memory.
2.2 Full OS voice tools like Talon
Talon is extensible system for controlling computers:
People who are weary of using AI copilots will find Talon to be a welcoming alternative. It’s entirely native, no AI.
I haven’t experimented with it yet, but I’m very interested in potential. You can connect Talon to AI using scripts, letting you use native solutions where it makes sense and using AI where it doesn’t.