vocalization

Vocalization (from now on just voc) is a speech to text to speech program.

The process it uses is as follows:

1. Loads a whisper.cpp compatible model (available from https://huggingface.co/ggerganov/whisper.cpp)
2. Initializes access to espeak
3. Live listens to the microphone, with the settings from config.json taken into account (live-reload)
4. Sends the audio to whisper to decode
   • If there is >=~1s of audio, then it goes through as normal
   • If there is <~1s of audio, then it gets padded with silence
5. The text from there is sent to espeak to speak out.

Future Goals

• Integrate a neural TTS (piper is the leading option)
• Output to a fake microphone instead
• UX improvements
• Better silence detection and noise suppression

Usage

It's evil, you shouldn't. ROCm only and probably requires some system libs that I can't list here because I already had them installed and don't know which ones they would be.

But if you want to:

1. Change the model path in the config.json file
2. Start the program and let it do its work

Extras

Yes I know I've committed the config. No I don't think it matters. ;3 There's nothing in the config that's useful to anyone outside of my home.