Xueliang Zhang’s research while affiliated with Northeastern University and other places

Enabling a single ring to recognize 3D keystrokes is a daunting challenge, especially for real-time keystrokes recognition. In this work, we integrated edge computing and machine learning algorithms into a microcontroller unit (MCU) of a smart ring so that keystrokes motion can be accurately recognized in real-time. We developed a multi-level decision (MLD) algorithm and embed a lightweight support vector machine (SVM) algorithm to execute computation for keystroke recognition on the edge of the smart ring. With this method, we can reduce the time for data transmission and avoid the data redundancy problem with the huge dispersion calculation workload; thus, improving the real-time performance of the smart ring device. Consequently, the application of this smart ring-based virtual keyboard system has minimum requirements for hardware, such as memory space and computing capacity. This study demonstrates that the use of low-performance chips in future virtual keyboard systems is possible in order to achieve lower development cost, reduced device size, and improved ease of use. The proposed smart ring is expected to provide a novel and convenient technology for real-world human-computer-interface applications in the future.

Virtual keyboards could potentially replace traditional physical keyboards for text input and human-computer-interface due to their portability and comfortable usage characteristics. However, typical virtual keyboards such as wearable keyboards and camera-based keyboards have unsatisfactory input recognition accuracy, and hence, preventing them from gaining pervasive acceptance by users. In this study, we demonstrated that the layout optimization of keyboards can potentially improve recognition accuracy for ring-type motion sensing virtual keyboards, and analyzed several layout designs for the configuration of typing keys. Moreover, we proved that the new layout designs have significant advantages in reducing muscle oppression for the users. The layout designs were mainly improved through three aspects: arc and split design, key position adjustment, and letter arrangement. Three new layouts were tested, which show significant improvements ranging from 5.74% to 7.51% in recognition accuracy, i.e., achieving up to 97.57% accuracy for ring-type virtual keyboards. Additionally, muscle oppression test results (using surface myoelectricity data) indicate that our new keyboard layouts could provide a significant difference ( $\text{p}< 0.05$ ) in reducing muscle oppression from traditional QWERTY keyboards. As for the moving distance of finger tips, our customized layouts could even reduce half of the movement distance compared with traditional keyboards. The experimental results demonstrate that our motion sensing rings combined with customized keyboard layouts could greatly overcome several disadvantages of traditional QWERTY keyboards.

In table tennis, a 40 mm in diameter table tennis ball weighing 2.7g can reach the opponent’s table in a very short time when it travels at a speed of 17m/s. That is difficult for new players to hit accurately. The purpose of this work is to recognize hits and misses by distinguishing the action difference between hitting and missing table tennis. The result of recognition can be used to figure out a small deviation from the correct posture and improve the hitting accuracy. Six volunteers participated in the experiment. The volunteers wore wristband sensors on the wrist on which they held the table tennis bat. Each volunteer played the ball 100 times and collected the information through a wristband sensor. The collected information was analyzed by support vector machine (SVM), decision tree (CART), linear discriminant analysis (LDA), K nearest neighbor (KNN), and Naive Bayes (NB) to identify the identities of volunteers and the hit or miss of the ball. The accuracy rate of volunteers’ identity recognition is 99%, and the accuracy rate of hits and misses is 95%. The results show that the wristband sensor can accurately identify the volunteers’ identities and missing cases through appropriate classification methods. This shows that we can find out the nuances between the stroke postures by hitting or missing the ball, and then use the results to correct the movement of the players, and thus improve the players’ skill.

In this paper, we present an improved ring-type virtual keyboard scheme that can achieve impressive performance with only one smart ring on a finger of each hand. The smart ring integrates a 6-DoF Inertial Measurement Unit (IMU) and a 3-DoF magnetometer sensor for collecting motion data during typing. First, a new keyboard layout is designed, by changing the previous rectangular layout to an arc structure, this method increases the difference in attitude angle between adjacent keys, which greatly improved the keystroke recognition accuracy. Secondly, other than the attitude angle feature, we also adopt acceleration data, gyroscope data and magnetometer data to describe the subtle differences between different keystrokes motion. Then, feature importance evaluation and feature correlation analysis were used to select features with high contribution rate and low similarity to describe keystrokes. Finally, nine effective features were selected from the attitude angle and magnetometer data for the final keystroke recognition. By weighing the number of selected features, recognition speed and recognition accuracy of training models, the keystroke recognition speed can increase by nearly 4 times while ensuring 98.53% of the keystroke recognition accuracy. This new ring-type virtual keyboard input scheme has the advantages in portability, small volume, and lower cost over many existing human-computer interface methods.

In this paper, we present a new scheme for implementing virtual keyboards, which uses only two to four motion-recognition rings per hand and a two-dimensional keyboard template (e.g., an A4 size paper with printed key positions). It has the benefit of portability, customizability, and low-cost when compared with existing approaches. Essentially, we have shown that wearing two wireless IoT rings on the middle phalanges of two fingers of each hand, users can input the alphabetic characters into a computing device by typing on a flat paper on a desk, and potentially in mid-air. We have demonstrated that two rings are sufficient in capturing the gestures and motions of all fingers in a typing hand for keystrokes recognition. A single wireless IoT ring, which weighs 7.8 grams, consists of a Bluetooth low energy (BLE) unit, a micro inertial measurement unit (mIMU), and a cell battery. The 3-axes attitude angles and the Z-axis acceleration of each ring are adopted as the features for keystroke recognition. The overall keystroke recognition accuracy rate can reach as high as 94.8% when two IoT rings are worn by a user on each hand; this accuracy rate increases to 98.6%, when four rings are worn on each typing hand.