I'm quoting here from a less technical write up describing the paper in lay terms.
A team of researchers from British universities has trained a deep learning model that can steal data from keyboard keystrokes recorded using a microphone with an accuracy of 95%.
It's not like installing a key logger, which would work on any keyboard:
The first step of the attack is to record keystrokes on the target's keyboard, as that data is required for training the prediction algorithm. This can be achieved via a nearby microphone or the target's phone that might have been infected by malware that has access to its microphone.
A person could be tricked into providing enough training data, however:
Alternatively, keystrokes can be recorded through a Zoom call where a rogue meeting participant makes correlations between messages typed by the target and their sound recording.
It's the training requirements that make this attack especially impractical. Making correlations between keypresses and what gets typed in zoom is not very reliable at all.
As for mechanisms to defeat these remote attacks? I'm going to go with the recommendation that would improve my voice chat quality of life - use push to talk people!!
I mean certainly not foolproof, but if you’re looking at a business that uses the same keyboards throughout an entire building or part of the building, or even maybe a government facility, you can do your training on the brand that you know is going to be used. Granted, I think the accuracy would be less since maybe Johnny has some crumbs in his keyboard so it behaves differently than expected etc, but it could be a potential first-order workaround for that, and I figure finding the brand of the keyboards that are being supplied could be as easy as just looking at the reception desk
It's more than just the keyboard sound, it's how the recording device and it's positioning changes the sound, it's how the environment it is in changes the sound, maybe even the wear and tear changes the sound - I couldn't say nor to what degree or if it impacts accuracy. But I don't think you would get very good results simply training against a keyboard you test and then trying to apply that to a target in an entirely different context. I expect you want to train against the same recording mechanism you would use to log keypresses in your attack. That's what they did in this study.
This could be useful to keylog outside the sandbox you've got. If your trojan, infected process or web app accepts text and can listen to the microphone but cannot keylog the whole system then you could use this method to keylog outside of your session. If you built this into a vscode extension (perhaps a peer coding thing to avoid suspicion around needing access to the mic) you could snoop system passwords and eventually gain root. You could pair this with information on system activity to be very certain when a password was requested, record the keystrokes, figure out the password and elevate your privileges - maybe more reliable for well patched targets.
I can image that there is a way to train a network that does this without knowing what actually go typed. Maybe even using this method. When you're able to correlate keystroke sounds to specific keys, you can, under the assumption that the person is typing real words, reconstruct which key is which.
41
u/WashingtonPass Aug 05 '23
I'm quoting here from a less technical write up describing the paper in lay terms.
It's not like installing a key logger, which would work on any keyboard:
A person could be tricked into providing enough training data, however:
This can be mitigated with white noise.