Date of Award
Master of Science in Electrical Engineering (MSEE)
Electrical, Computer, and Biomedical Engineering
Since the invention of the computer mouse interface there have been no major changes in its design. The way we interact with computers through keyboards and mice are still the same as they first were. Recent new technologies try to replace or emulate the mouse and each have their own disadvantages. Some require the use of surfaces, cameras, sensor bars, or tethering. Other technologies are gesture based, require training, and are not intuitive. Not only is there the need for a mouse interface that is intuitive and convenient, but advancements in the field of human computer interfacing, especially using electromyogram (EMG) sensors, further open doors. The objective of this project is to design and create a wearable computer mouse interface, independent of cameras, sensor bars, surfaces, or tethering, that is intuitive, fast, accurate and convenient. This is accomplished by combining the output of both EMG sensors and an inertial measurement unit (IMU) to control the cursor and clicking functions of a mouse. The goal is to be intuitive and accurate in order to overcome the shortcomings in existing technologies.
Many different muscles were examined as potential candidates for the EMG sensors. The output of each sensor was observed and several different sensor setups were chosen for focus. For each setup, at least one classifier configuration was designed in the attempt to obtain the highest testing accuracy. The angular velocity measurements of the z and y axis in the IMU were chosen to physically control the mouse cursor. Several different techniques were designed based on the different sensor setups and classifier configurations to obtain the highest overall system accuracy and fewest false positives.
Training data was collected in MatLab for processing. Time domain features were extracted and provided as input to a linear discriminant classifier. The implementation of the designs were programed in C++ for fast and efficient real-time performance. The setup and configuration with the highest training accuracy, highest testing accuracy, lowest false positive rate, and best overall IMU integration, was the setup with EMG sensors on the muscles: Extensor Carpi Radialis, Extensor Carpi Ulnaris, Flexor Carpi Ulnaris, Flexor Carpi Radialis, and Flexor Digitorum Superficialis (Near Wrist). One classifier took the input of all five channels for classification into a total of seven classes or actions: Rest, Up, Down, Left, Right, Left Click, and Right Click. This design took the least time to complete the accuracy program designed to evaluate the different configurations. This design also saw the lowest false positives.
Compared to a standard mouse or track pad, the best design still fell far short in performance. However, the common user has been using a standard mouse or track pad their whole life, and the need for time to practice with the device is expected. With practice, the design has the potential to match or come close to the accuracy of existing mice. With further automation of the training data collection and classifier training, the device would require little configuration to start using. Further development towards a commercial product would minimize the equipment required, the sensor setup time, and make the device no less convenient than a standard mouse.
One of the design goals at the forefront of this project, is the application and usability by hand amputees. Muscle locations are located in the forearm, and the IMU device located on the back of the wrist or forearm, allows hand amputees and those with hand disabilities to operate the device.
Forbes, Timothy, "Mouse HCI Through Combined EMG and IMU" (2013). Open Access Master's Theses. Paper 43.