Mouse HCI Through Combined EMG and IMU

Since the invention of the computer mouse interface there have been no major changes in its design. The way we interact with computers through keyboards and mice are still the same as they first were. Recent new technologies try to replace or emulate the mouse and each have their own disadvantages. Some require the use of surfaces, cameras, sensor bars, or tethering. Other technologies are gesture based, require training, and are not intuitive. Not only is there the need for a mouse interface that is intuitive and convenient, but advancements in the field of human computer interfacing, especially using electromyogram (EMG) sensors, further open doors. The objective of this project is to design and create a wearable computer mouse interface, independent of cameras, sensor bars, surfaces, or tethering, that is intuitive, fast, accurate and convenient. This is accomplished by combining the output of both EMG sensors and an inertial measurement unit (IMU) to control the cursor and clicking functions of a mouse. The goal is to be intuitive and accurate in order to overcome the shortcomings in existing technologies. Many different muscles were examined as potential candidates for the EMG sensors. The output of each sensor was observed and several different sensor setups were chosen for focus. For each setup, at least one classifier configuration was designed in the attempt to obtain the highest testing accuracy. The angular velocity measurements of the z and y axis in the IMU were chosen to physically control the mouse cursor. Several different techniques were designed based on the different sensor setups and classifier configurations to obtain the highest overall system accuracy and fewest false positives. Training data was collected in MatLab for processing. Time domain features were extracted and provided as input to a linear discriminant classifier. The implementation of the designs were programed in C++ for fast and efficient real-time performance. The setup and configuration with the highest training accuracy, highest testing accuracy, lowest false positive rate, and best overall IMU integration, was the setup with EMG sensors on the muscles: Extensor Carpi Radialis, Extensor Carpi Ulnaris, Flexor Carpi Ulnaris, Flexor Carpi Radialis, and Flexor Digitorum Superficialis (Near Wrist). One classifier took the input of all five channels for classification into a total of seven classes or actions: Rest, Up, Down, Left, Right, Left Click, and Right Click. This design took the least time to complete the accuracy program designed to evaluate the different configurations. This design also saw the lowest false positives. Compared to a standard mouse or track pad, the best design still fell far short in performance. However, the common user has been using a standard mouse or track pad their whole life, and the need for time to practice with the device is expected. With practice, the design has the potential to match or come close to the accuracy of existing mice. With further automation of the training data collection and classifier training, the device would require little configuration to start using. Further development towards a commercial product would minimize the equipment required, the sensor setup time, and make the device no less convenient than a standard mouse. One of the design goals at the forefront of this project, is the application and usability by hand amputees. Muscle locations are located in the forearm, and the IMU device located on the back of the wrist or forearm, allows hand amputees and those with hand disabilities to operate the device.


LIST OF TABLES
Advancements have been made in electromyography that allows readings to be taken from electrodes on the surface of the skin. Surface EMG sensors are easy and fast to apply to the skin. This technique is also much safer, allowing everyday users to measure muscle activity. These devices make human computer interface much more practical. However, not all muscles are located adjacent to the skin. Some are located underneath other muscles or bone, limiting the number of muscles available.
Specifically, an EMG sensor measures the electrical activity in a muscle. The voltage measured is on the order of millivolts. Before processing on a computer, the data must first be converted from analog to digital. Each A/D device has an input voltage requirement, usually around -5 to 5 volts. Thus, an amplifier is used to scale up the signals from the EMG sensors to meet this requirement.
With the popularity of both the personal computer and the mouse as the primary interface, several areas of the population are put at a disadvantage. The elderly and the disabled are often limited in their ability to use a mouse, and thus a computer. The new advancements of surface EMG sensors allow for the measurement of wrist and finger actions through muscles in the forearm. This gives hand amputees the potential to control a mouse through such an interface.
An inertial measurement unit (IMU) measures acceleration, angular velocity, and gravitational forces on all three axis by the use of accelerometers, gyroscopes and magnetometers. They are inexpensive and accurate in measuring motion, making them very practical. IMU's are very common in many technologies including aircraft, spacecraft, boats, satellites, and missiles. Because of their size, they can easily be included in embedded systems and small devices. They often include built in software to compensate for erroneous constant measured motion, called drift.
Many other technologies have been developed in the attempt to replace or emulate the functionality of a mouse. A paper from 2001 details a design based solely on an inertial measurement unit, where sufficiently small acceleration is considered cursor movement and larger movements are mapped to functions such as clicking (Lee). Another design is to detect and track user eye movement patterns and control mouse cursor movement (Miyoshi). The field of human control interface using electromyography is relatively new. Advancements in the field have improved the accuracy to the point where computer interfaces become practical, and feasible. EMG signals have also been used in a variety of applications, including an EMG-based power assisted wheelchair (Oonishi), and neural controlled artificial arms (Englehart).
In order to classify actions within the EMG sensor data, pattern recognition techniques are required. Pattern recognition is the classification of input data to existing labels. In the case of this design, the labels are the separate wrist movements and finger clicking movements. Common methods for classifying EMG data are by time-domain analysis, where a set length of time of EMG data is analyzed for a pattern, and future data compared to those patterns. Specifically, common timedomain features are thresholds, zero crossings (crossing over from negative to positive or vice-versa), and slope sign changes. Common classifiers for EMG analysis are linear discriminant analysis, neural networks, fuzzy logic, and support vector machines (Al-Timemy; Khezri; Khushaba; et al). Each classifier must extract these features from the appropriate and relevant EMG sensors.
Design challenges focused around a computer mouse interface include accuracy, intuitiveness and convenience. A lack of any of these three makes the device impractical and inferior to the existent standard mouse design. Many devices are simply not accurate enough. The slightest inaccuracy becomes a major inconvenience to the user, considering a device as widely and commonly used as a mouse. If the proposed replacement to a mouse is not accurate, then a user may spend hours practicing with the device in order to achieve a level of accuracy that will still never match a conventional mouse. Lastly, convenience is a big factor, as a user will not want to use the device if it requires an hour setting up. In this paper, a novel human computer interface is proposed by combining the output of both EMG sensors and an IMU device.

OBJECTIVE, HYPOTHESIS AND SCOPE
The objective of this thesis is to design and create a wearable computer mouse interface, independent of cameras, sensor bars, surfaces, or tethering, that is intuitive, fast, accurate and convenient. Combining the output of both EMG sensors and an IMU to control the cursor and clicking functions of a mouse should be intuitive and accurate, overcoming shortcomings in existing technologies, and also allowing hand amputees to operate the interface. Hand amputees are disadvantaged for both traditional mouse and typing actions. This project specifically focuses on the mouse interface.
Most existing technologies and designs listed in the review of literature section below are gesture based. One of the main goals of this design is to be intuitive, which is realized in the design of this project as a non-gesture based interface. Detecting normal mouse functions through EMG sensors is difficult. It is hard to distinguish a clicking action between each finger, and even wrist movements. This leads to most interfaces being gesture based, where gestures are chosen that are easily distinguishable from the rest. To overcome the difficulty in distinguishing these actions, the output from both the IMU and EMG sensors will be combined. How this is implemented will determine the balance between accuracy and intuitiveness. 5 The scope of this thesis is limited to a proof of concept. If the design meets all the desired objectives then the next step may be to develop a product. The realization of a product involves measures beyond the scope of this thesis, including but not limited to: acquiring greater sources of funding, and design and fabrication of hardware based on the prototype. Further testing and development would be required to translate the prototype to a product. The potential of a product was taken into account in the design of the prototype. EMG muscle locations were chosen for their ability to be incorporated in an arm band style sensor array, to allow for a convenient and fast setup process.

REVIEW OF LITERATURE
The principal literature serves as both a timeline of computer mouse interfaces as well as the current research in human computer electromyography control. The history of mouse interfaces depicts the advantages and disadvantages of each device.
The research in EMG measurement, control and interface, details existing classifying techniques, muscle target locations, accuracy and error rates, and progress in the field.  (Cheng). Another patent is of a glove with pressure plates on two fingers for left and right mouse clicks. Scrolling buttons are located on the side of the index finger for use by the thumb. There is a side switch to turn off the device for using your hand for other things and a tracking device on the tip of one of the fingers (Bajramovic).
A general web search led to several more devices with the same variety of characteristics as the patent search. One device, called the G-speak, is a spatial operating environment. It is a user interface and networked system that allows users to control a variety of things, including controlling data and objects in 3D, based on gestures (Oblong Industries). It operates by use of several cameras that map your hands in 3D. The system is the glove interface from the movie Minority Report, created due to its popularity in the movie. The Mister gloves, a wireless USB gesture input system developed by two students from Cornell, is a gesture-based glove designed to replace the mouse and also include several other hotkeys or shortcuts. It uses accelerometers and gyroscopes to map gestures to mouse movements or hotkeys.
The glove is wired to a base station that communicates wirelessly with the computer (Chen). Another device, called Leap Motion, is another gesture based device that replaces the mouse. It works by a motion sensor placed on the

ANALYSIS OF LITERATURE
An important observation to note is how the accuracy is measured throughout the various publications. They report classification or recognition accuracy, which is the fact that the user is performing the action and not the amount or magnitude of that action. Because of this, many accuracies reported are deceptive when stated relative to the goal of a mouse interface. For example, recognizing the wrist up movement does not indicate how much the mouse cursor should move up. The use of an IMU, as realized in the last paper cited above, accomplishes this. However, that paper accomplishes both IMU cursor movement and clicking functions through the use of gestures. The movements performed are not the same as they would be using a conventional mouse. This makes the interface not intuitive and requires much learning and practice. A trend throughout the papers is the selection of the pattern recognition classifier. All papers use some version of linear discriminant analysis or neural network. Another useful fact to be noted, is that the use of a constraint improves recognition accuracy. Therefore, some sort of constraint may be used to improve the design proposed in this project.
Of all published work and products created, the design or device that is most similar to that proposed and tested in this project is the Myo device. This device was announced mere weeks before the writing of this thesis, and is not expected to fulfil preorders until the end of this year, 2013. While the technology is similar, the purpose of the device is still different from what is examined here. The application of the Myo appears to be app based. The promotional video shows a user waving their hand and a music player switching to the next song, or a video game where the user's hand assumes the position of a gun and performs gestures to simulate firing and reloading.
The design of the device seems to include the functionality necessary to act as a mouse interface, although, that is not its purpose and is never mentioned. As the release of the device is still far away, the description is vague. Furthermore, the intuitiveness is difficult to determine through the promotional video, and due to the fact that it is gesture-based. With all of that in mind, there remains a place for the design proposed in this project, while remaining unique.

MATERIALS AND METHODS
There were many steps required to both design and test the system. To determine if the idea of a combined EMG and IMU mouse interface was feasible, both EMG and IMU data needed to be collected and analyzed. Based on the output, further designs could be developed and tested. The first was to determine the relevant and optimal data to use from the IMU. Secondly, the wrist and finger actions needed to be determined in order to determine the appropriate EMG muscle locations. For the EMG data, a feature extraction and pattern recognition algorithm must be chosen to classify the data. Once this is done, the method of combining the two outputs can be determined. Lastly, a method is required to determine the overall accuracy and intuitiveness of the design.
An inertial measurement unit measures acceleration, angular velocity and gravitational forces on all three axis, by the use of accelerometers, gyroscopes and magnetometers. The wireless IMU system in the lab is the Xsens MTx. It is made by Xsens and contains an API to read the data from within MatLab and C++. It is advertised to not have any drift, which is defined as a small constant movement measured when the device is stationary. The IMU is tethered to a battery powered pack that wirelessly communicates to a small antenna, connected to the computer via USB.
In the design, the IMU will be used to control the mouse cursor. Upon viewing the output from all three of the accelerometers, gyroscopes and magnetometers, the angular velocity output from the IMU matches physical wrist movements the best. The acceleration and magnetometers provide useful data but are not pertinent to cursor movements. The orientation of the IMU on the back of the wrist determines which axis are relevant. With the IMU on the back of the wrist and the arm flat, the IMU is oriented so that the z-axis runs vertical, the y-axis horizontal and perpendicular to the arm, and the x-axis parallel to the arm. Figure 3.1 visually depicts this. This results in z-axis gyroscope data translating to the mouse y-coordinate, and y-axis gyroscope translating to the mouse x-coordinate. X-axis data can be ignored. Figure 3.2 depicts this visually. The gyroscope data are measured as angular velocity. Both angular velocity and linear velocity affect the sensors in the same way, making the prospect of an IMU as part of a mouse cursor control system more versatile.  There are two EMG systems available in the lab, a wired and a wireless system. and finger movements associated with it to prevent false classifications. Table 1 shows the muscles associated with the wrist to be examined. Table 2 shows the muscles associated with finger movements.  All observations for the four locations listed in Table 1 matched the given test maneuver and do so with little crosstalk. The ideal EMG sensor locations on all four muscles are also conveniently located a third the way down the arm from the elbow.
The test maneuvers also correspond directly to the desired actions of the mouse cursor.
The muscle locations and accuracy observed for the fingers in Table 2 were not as conclusive as those of the wrist muscles. The muscle location and accuracy determines what movement or gesture will be considered a mouse click. To be intuitive, the click gesture should resemble the click motion on a convention mouse. That eliminates the use of any muscle used for finger extension or thumb movement. Another reason to eliminate thumb muscles is due to the small size and difficult sensor placement for those muscles. Bending just the tip of a finger is not very feasible and making a fist is not very specific, making the Palmaris Longus and Flexor Digitorum Profundus muscles not practical. The Palmaris Longus is also too small of a muscle to accurately read. The Flexor Digitorum Superficialis is the most relevant and optimal muscle to use. However, significant crosstalk was observed from the Flexor Carpi Radialis, the muscle associated with the down and left action by a right hand. To compensate for this, the location of the sensor can be moved up towards the wrist or down and to the outside of the arm near the elbow (Technical Note 101: EMG Sensor Placement 5).
There are five possible EMG sensor locations to use in our pattern recognition classifier. The five muscles are: Extensor Carpi Radialis, Extensor Carpi Ulnaris, Flexor Carpi Ulnaris, Flexor Carpi Radialis and Flexor Digitorum Superficialis. In order to know which action the user is performing, the output from either all or a select few of the sensors need to be measured and compared against predefined actions to form a prediction. There are numerous ways to combine these sensors and numerous predefined actions we can record. Each combination will have a different degree of accuracy. Before we can form the classifier, the data from the EMG sensors need to be quantified. This is called feature extraction and identifies properties or attributes in the data. The purpose of discriminant analysis is to classify observed data to a defined class or label in which the posteriori probability is the greatest. The conditional probability is the probability of class Cg given the observed feature vector ̅ is: where: G = number of classes = set of classes, where [ ] ̅ = feature vector of the given window of samples = priori probability ̅ = likelihood (conditional probability) ̅ = probability of the observed feature vector ̅ The user is capable of performing any action at any time, so we assume our priori probability to be the same for all classes. Thus, every class's covariance is equivalent, and the maximization of the posteriori possibility is:

{ ̅ }
The linear discriminant function is defined as: and have the same number of columns. With the mean vector now known, the linear discriminant function above becomes: In real time testing, the observed feature vector ̅ from each window of samples is inputted into the classifier above for each class. The prediction made, ̃ , satisfies: Furthermore, to eliminate sudden and limited incorrect decisions from the classifier, majority vote is used. The increased accuracy, however, comes at the cost of a delay in the response time. The size of the majority vote window will be varied and the resulting accuracy and delay results observed.
The system to improve accuracy and response time proposed in Englehart's paper is the overlapping window continuous decision stream. A large window of time is needed to extract features from the data, and to supply the classifier. This slows down the prediction rate of the classifier and hinders real time performance. The system proposed involves the window length, which is calculated every set interval, defined as the window increment. If the window size is 150 milliseconds, a window increment can be 50 milliseconds. Figure 3.3 shows the overlapping window scheme.
The window increment in this design shall be 20 milliseconds, to ensure a quick and real-time decision stream. The window length will also be kept short, but can be varied and the accuracy of the classifier observed to determine the optimal length.

PATTERN RECOGNITION CLASSIFIERS AND COMBINED OUTPUT
The main concept in the design is to combine the sensor output from both the EMG and IMU sensors. The combination of both outputs will be used to control the mouse cursor while the mouse clicking functions will be controlled solely by the EMG output. One primary feature of the design is to detect and ignore false positives. False positives are defined as any action the user may perform in everyday life that is not meant to be translated to a mouse operation. Another objective of the design is to be accurate enough to be a replacement for the standard mouse; a goal not yet obtained by any current design or device. A block diagram of the entire system is given in figure 3.4. Figure 3.4. Block diagram of the entire system. This is an overview of the EMG and IMU mouse interface system. Values from Train Classifier (offline) are required in Real-time Testing, therefore Train Classifier is performed before Real-time Testing.
The mouse clicking functions will be implemented through solely EMG sensors. Both left and right click actions need to be identified. The functionality of pressing and releasing the mouse button, as it is known on a standard mouse, need to be separate functions. This enables drag and drop functionality. The EMG muscle locations selected for these functions need to differentiate between the index and middle finger, as well as be independent from wrist movements.
For mouse cursor movements, the IMU will be the primary input source.
However, the IMU data are highly sensitive to movement, making it difficult to keep the cursor stationary when no movement is desired. Unintentional movement is defined as a false positive. To counteract false positives, the EMG classification part of the system is introduced. The EMG is used to know which direction the user's wrist is pointing in order to know which direction they desire the IMU output to be going.
The EMG part of the design can effectively be called an error or false positive detector.
The basic design is to combine the two outputs by only accepting the output from the IMU when the output of the EMG agrees with the IMU's direction. This prevents false positives and would allow the user to move their arm for everyday activities and not move the mouse cursor during such movements. Any movement above a certain threshold and below another threshold will be considered zero, or no movement. This prevents jitter, drift or any movements too small to be desired as cursor movements to register. Movements that are too large are ignored for the same reason.
Because the IMU is the primary input for cursor movements, and the EMG classifier works to prevent false positives, it leaves the specific design of the system very open. The accuracy and precision of just the EMG output is not as important as the final system. Creating too precise of an EMG classifier has the potential to decrease the accuracy and usability of the finished product. This results in flexibility in the choice of muscle location and the formation of classifiers. However, since click functionality is controlled solely by EMG, we need to ensure that the sensor and classification of clicks is as accurate as possible, and the sensor location takes priority over wrist muscle locations.
The four locations for the wrist each represent a different direction, and contain little crosstalk. However, they may not all be necessary. Depending on where the electrode is placed for the Flexor Digitorum Superficialis (F1), the crosstalk varies. If it is placed closer to the wrist, the crosstalk is minimal, if it is placed half way between the elbow and wrist, then there is crosstalk from the Flexor Carpi Ulnaris (W2), the muscle associated with the down and right action by a right hand. Therefore, I propose three different configurations for the EMG sensors. Each configuration and their corresponding classifier options are listed in Table 3. The first is all five locations, the four for the wrist and single sensor closer to the wrist for clicking functionality (Setup Communis and W1 are located adjacent to each other, and contain a great amount of crosstalk. It is infeasible to place both sensors next to each other with the hope of obtaining distinguishable signals. Therefore, one sensor will be used on W1. Because of the second sensor on the F1, classification accuracy for clicking should remain high.    three classes for wrist movements. This helps prevent false positives in the clicking classifier, and simplifies the wrist direction classifier to obtain higher overall accuracy with the IMU.
For each setup and configuration, the accuracy and percent error will be calculated to determine the optimal configuration. Each setup and configuration will be tested in real time to determine the accuracy when combined with the IMU output.
To help improve the accuracy of the design, limit the effort required by the user, and prevent hyper-extension of the wrist, a simple wrist or glove type of constraint will be used. The current restraint used is the Ace Tek Zone wrist brace. However, to meet the goal of being convenient to wear and use, it must not interfere with any everyday action the user might want to perform.
The fusion method for combining the EMG and IMU output varies based on the sensor setup and classifier configuration. For all cases, the IMU output is only accepted when the direction of the output agrees with the direction classified from the EMG data. The following directions given correspond to wrist/hand movements, rather than the coordinates of a computer screen, where the y-axis is inverted. For Setup A, Configuration 1: When the prediction is the direction Up, IMU output is only accepted if the y direction is positive/up and the absolute value of the slope is greater than 1, forming a v-shaped region of accepted values.     milliseconds. The window increment for the EMG data has been chosen to be 20ms.
Thus, the IMU must be sampled twice per EMG prediction. The window length will be determined by accuracy and delay in classification, and will initially be set to 60ms.
In MatLab, the A/D is called by using the built in data acquisition toolbox. An analog input is defined and number of channels and sample rate set. Data is collected by a simple function call that returns the newest window increment size of data in the A/D buffer. The MatLab code for data collection, feature extraction and the LDA training algorithm are given and detailed in Appendix A. In C++, the size of the buffer in the A/D must be allocated by specifying the multiplying the number of channels with the window length size. Another FIFO buffer is set up, first in first out, to store each window increment up to the size of the window length, to be used later in classification. This is done by repeatedly calling a function that returns the index, or the current number of samples collected per window length, to know that another window increment has been collected, and it can be copied into the FIFO buffer. For each window increment where a full window length of data are available, the data are passed to the LDA classifier, explained in a previous section. The feature extraction, LDA classifier, and majority vote function are written in C, with efficiency being the priority. The size of the majority vote decision stream is set to 20, and is varied to observe what size delivers the optimal accuracy per delay. The decision outputted by the majority vote function for each classifier forms the EMG portion of the program.
As mentioned earlier, the pertinent data from the IMU are the z-axis gyroscope and y-axis gyroscope, translating to the mouse y-coordinate and x-coordinate, respectively. The axis data proportionately corresponds intuitive hand movements to cursor movements. All that is needed is appropriate scaling of the values. The level of scaling in order to match hand movements appropriately to the screen depends on both the screen size and the distance of the user to the screen. The units of the IMU data are meters per second. An appropriate scaling factor found for the y-axis is 36, and x-axis 42. This forms the IMU portion of the program. perform a set number of clicks at predefined locations. A graphical user interface was created with circles located in the window and a predefined order in which the user clicks inside them. Along with the elapsed time, the number of incorrect clicks performed during the trial is also recorded. There is a total of nine clicks the user must make and each click outside the specified circle will count as an incorrect click. The user will perform the task with a standard mouse and then with the EMG and IMU mouse interface. The elapsed time and incorrect number of clicks will then be compared. The program was written in C++ with the Win32 API. Figure 3.10 shows a screenshot of the program. Figure 3.11 shows a screenshot of the program while it is running. Appendix C contains the code for the program.
The goal of the project is to be intuitive and convenient. This also means that setup time as well as ability to wear the interface while performing everyday tasks is a factor in the accuracy and effectiveness of the design. However, the scope of this project is limited to a proof of concept, so setup time and ability to wear the device are not as heavily weighted in the accuracy of the design. Furthermore, an observation of false positives when wearing the device and performing everyday tasks, such as typing on a keyboard or writing with a pencil, will be noted towards the overall accuracy of the device.   Motion artifacts are noise in the EMG sensor generated by movement, either physical movement or vibrations, or movement against the skin. This is generated by the wrist brace, which is located right next to the sensor. For future data collection and testing, the sensor was moved slightly farther away from the wrist brace. For finger clicking actions, some crosstalk is detected in W2, which was anticipated. However, F1 is a prominent muscle for these actions compared to the others. A short spike in the voltage is observed for brief left and right clicking actions. Left press and right press represent continuously holding out the left and right click actions, respectively. A clear difference between the voltage amplitudes of the two actions is observed.
The actual training data recorded for use in the training of the LDA classifiers were taken by holding out each action for several seconds. For each action, features were extracted and used for training the classifiers. The training error is calculated for each classifier of each setup and configuration. The training error is calculated by the percentage of points that are incorrectly classified, and fall on the wrong side of the decision boundary formed by the classifier.  In real-time, each setup and configuration was tested. The real-time EMG decision was observed while each action is performed to note the accuracy and occurrence of false positives. The accuracy of the classifier is defined as how often it classifies the action correctly. False positives are defined as incorrect actions given for that classifier when the actions of a different classifier are performed. There is no way of calculating the testing error, as the class the action belongs to is decided by the user in real-time. Therefore, a rating system was developed, as listed in    In table 4  are present in F1. They were generated by the wrist brace being worn, which was located next to the sensor. For future data collection and testing, the sensor was moved slightly farther away from the wrist brace.   Compared to a standard mouse or track pad, the best design still fell far short in performance. However, the common user has been using a standard mouse or track pad their whole life, and the need for time to practice with the device is expected.
With practice, the design has the potential to match or come close to the accuracy of existing mice. With further automation of the training data collection and classifier training, the device would require little configuration to start using. Further development into a commercial product would minimize the equipment required, as well as the sensor setup time, to make the device no less convenient than a standard mouse.
One of the design goals at the forefront of this design, was the application and usability by hand amputees. Muscle locations are located in the forearm, and the IMU device located on the back of the wrist or forearm, allows hand amputees and those with hand disabilities to operate the device.
As with any design, there are many improvements and new ideas that can be tested and added to increase the accuracy and further minimize false positives. The proof of concept of this project was a success, and is part of a bigger and growing field of research. The applications and potential of this design and research are large.

FURTHUR DEVELOPMENT
There will always be more features to add and different methods to try in an attempt to increase accuracy and usability. Further functionality can be added to emulate mouse wheel scrolling or middle clicking. As computers and device slowly switch to a more gestured based interface, such as swiping to scroll vertically or horizontally, could be added.
The original plan was for the design to be implemented on an embedded system. With an embedded system, all calculations would no longer be performed on the actual computer. This would allow the device to work as a normal mouse, simply sending the mouse related commands to the computer. The requirements for the embedded system are as follows: be able to run C++ programming code, be able to communicate wirelessly through Bluetooth, contain an A/D, have an interface for EMG sensor signals, and contain an IMU. Research was done and determined that all functionality is met with the combination of the Gumstix Overo Air COM computeron-module board and the Gumstix RoboVero LPC1769 microcontroller board.
Once the embedded system implementation is completed, custom hardware could be manufactured to turn the board into a fully functional commercial product: small, stylish, easy to set up and use, and inexpensive.  1,j).feat,2))]; end N = size(TrainData.F,1); Ptrain = size(TrainData.F,2); %training data size sc = std(TrainData.F(:)); %training data + noise TrainData.F = TrainData.F + sc./1000.*randn(size(TrainData.F));   This is the main file for the Real-time implementation of the EMG IMU mouse interface. It is the code unique to Setup A, Configuration 3. This file is organized in the following way: First, the LDA classifier values are loaded from text files that were created and exported from MatLab. Next, the A/D for the EMG channels is initialized, followed by the IMU device. Then our main loop initializes data collection, collects data, classifies the EMG data, and combines the output of both EMG and IMU to form our design.

APPENDICES
Both the A/D and IMU initialization uses the API that comes with the devices. The code is based off the example files and documentation that also accompany the device and device software. The size of the buffer in the A/D must be allocated by specifying the number of channels and the window length size. Another FIFO buffer is set up, first in first out, to store each window increment up to the size of the window length, to be used later in classification. This is done by repeatedly calling a function that returns the index, the current number of samples collected, to know that another window increment has been collected, and it can be copied into the FIFO buffer. When the buffer is full the index wraps around to the beginning and continues collecting data. For each window increment where a full window length of data are available, the data are passed to the LDA classifier.
The feature extraction, LDA classifier, and majority vote function are written in C, with efficiency their focus. The code for feature extraction and the LDA classifier is shown below: EMG_PR.cpp. The size of the majority vote decision stream is set to 20. The decision outputted by the majority vote function for each classifier forms the EMG portion of the program.
The pertinent data from the IMU are the z-axis gyroscope and y-axis gyroscope, translating to the mouse y-coordinate and x-coordinate, respectively. The units of the IMU data are meters per second. The scaling factor used for the y-axis is 36, and x-axis 42. This forms the IMU portion of the program.
Combining the data from the LDA classifier and IMU device is done according to figure 4.5. IMU data is only accepted when the slope agrees with the EMG action. The regions of accepted values from the IMU overlap the other regions. The absolute value of the slope is computed and for actions Up and Down, slopes larger than 3/4 are accepted. For actions Left and Right, slopes less than 4/3 are accepted. This forms the EMG and IMU fusion portion of the program.
The mouse function calls and API used in the Real Time C++ implementation is that of Windows, using the Windows.h header file. When updating the x and y coordinates of the mouse, we use the API to get the current mouse coordinates. Doing this allows other mouse input devices and programs to be used concurrently, the same way commercial mice operate. The clicking and functionality operates as follows: The last clicking action is stored in order to know what the next appropriate action to be taken is. For example, if the action is left mouse click, the left mouse press function is called. If the last action was a left mouse click and the next action is rest, then the left mouse release function is called. This concludes how the mouse functionality works.