Mobile devices, such as phones and wearables, include an increasing variety of more and more accurate sensors, only part of which the users can control to a certain extent to protect their privacy. In the meantime, mostly with respect to the accelerometer and gyroscope sensors of smartwatches, various keylogging side-channel attacks have been described in literature, demonstrating that sensitive information like passwords can be inferred from the data recorded by these sensors.
In this thesis, we take a closer look at the Myo armband, a wearable device worn on the upper part of the forearm containing an accelerometer, a gyroscope, a magnetometer and eight electromyographic (EMG) sensors for measuring muscle activity. In particular, we investigate whether the EMG data supports the recognition of finger movements sufficiently to detect new keystrokes of the same person or of previously unseen typists.
We create a dataset based on both keystroke and sensor data collected from 27 volunteers wearing two Myo armbands while typing on a physical keyboard. In order to detect keystrokes based on this data, we apply supervised learning approaches utilizing a random forest, a convolutional neural network (CNN) adaptation of WaveNet and a convolutional recurrent neural network (CRNN).
We estimate the predictive performance, achieving a mean f1 score of 0.75 for the CRNN in the within-subject scope and an f1 score of about 0.61 for the between-subject scope, independent of the chosen model. These estimates are validated in a proof of concept, achieving a mean f1 score of 0.64 for the CRNN in the within-subject scope and a mean f1 score of 0.65 for the WaveNet adaptation on an unseen person in the between-subject scope.