A Novel Magnetics and IMU Based Design for Untethered and Hands-Free Human Computer Interaction

The current status of Human Computer Interaction (HCI) stands to develop greatly, with many new technologies offering promising new methods interacting with a computer. The goal of this project is to develop a novel device that is compatible with desktop style point-and-click interfaces designed for mice but has the unique advantages of being ’Untethered’ and ’Hands-free’. The use of Electromyography (EMG) and an Inertial Measurement Unit (IMU) was initially investigated, leading to an alternative design utilizing a series of small Hall Effect sensors and corresponding magnet(s). The cursor response was identified as the most important factor to ease of use, with physiological tremor being the main source of disturbance. Without friction to cancel physiological tremor, countering algorithms were developed to improve ease of use. A final prototype was developed as part of testing and for proof of concept.

The current status of Human Computer Interaction (HCI) stands to develop greatly, with many new technologies such as computer vision promising new interaction with a computer. However, the most recent major breakthrough in HCI can be seen in the widespread application of touchpads and touchscreens for mobile computing such as in smartphones and tablets. However the standard computer mouse is still in widespread use for desktop computers. A possible explanation is that for a new device to enter mainstream use it must both be compatible with existing user interfaces, and provide a unique advantage to its use in those existing interfaces. A new device could prove its value using existing interfaces even though it could support more advanced methods of user input.
Later, if interest is garnered, more advanced interfaces could be developed to take advantage of the additional ability. The goal of this project is to develop a novel device that is compatible with desktop style point-and-click interfaces designed for mice, but has the unique advantages of being 'Untethered' and 'Hands-free'. A prototype device was created and tested which utilizes a combination of two types of sensors, an Inertial Measurement Unit (IMU) and a series of small Hall Effect sensors and corresponding magnet(s).
To further understand why new technologies may not be accepted, one can imagine a sensor that can track the user's hand and individual finger movements.
While such a device may get some attention from electronics media for being novel, there is no readily available interface in current use that could make effective use of such data. One could create a simplified design where a single finger is tracked as the mouse cursor, with finger flexing indicating clicking actions. However doing so would not take advantage of all the extra data that the sensor provides beyond what a normal mouse is capable of. Without the extra data being used the device does not provide any other strong merit to entice purchase.
On the other hand, one can examine how the touchscreen has entered widespread use. Especially for mobile computing, the touchscreen boasts a great advantage from being built into the screen and not being an external device that must be carried around in addition to the main device. Before modern smartphones reached markets touchscreens could be found in occasional use as effectively a multi-layer interactive button panel. However, because touchscreens had been widespread, but not common use, when modern smartphones were developed the use of a touchscreen became obvious. In addition, advancements in touchscreen technology making them easier to use ensured their adoption. However, as noted by Bogdanoff [1] in an article about the rise of touchscreens, "It is not enough to simply add a touch screen to an existing UI. Trying to navigate awkward menus and select small icons will quickly cause you to go back to the mouse and keyboard." Smartphone operating systems and applications have been designed from the ground up to support the touchscreen interface. Only now with their success are touch interfaces entering the realm of personal computing devices.

Objective and Scope
Therefore with these factors in mind the objective of this project is to create a new design for HCI that is compatible with the standard mouse interface, but provides advantages over the standard mouse. The device would also provide the possibility of more advanced input that could be exploited should interfaces to harness the additional functionality be developed. In terms of testing the performance of the device, it is difficult to provide a simple metric to judge performance that can be compared to existing devices.
The more important factor in evaluating the device's success is ease of use. One of the biggest challenges in developing the device proved to be disturbance in the user's pointing response resulting likely from Physiological tremor. Physiological tremor is a much lesser issue for a standard mouse because friction between the mouse and the surface it is placed on prevents weak disturbances from affecting the cursor position. This is important for most point-and-click interfaces as movement between the triggering of a mouse button press and release correspond to dragging.
Without friction to filter out small disturbances it is nearly impossible to trigger a simple click instead of a dragging input. Therefore the biggest factor is not the true accuracy of the device and performance which could be easily measured, but whether it provides a similar filtering effect while also feeling properly responsive. The idea of a glove mouse is not a new idea as there exist designs of this type already. A prime example is "Computer mouse on a glove" [1] which details a simple glove design that uses pressure plates and external sensor for tracking. This design is a tethered design due to the use of an external sensor for tracking the glove. Another example is the complex "Magic Mouse" [2] which details a doublehanded multi-component system including two different sets of inertial sensors, and a number of contact sensors. While this device is an untethered design as it uses sensors placed on the user, it is not hands-free because of the contact sensors which are bulky and cover the palm of the user's hand. The authors do mention the untethered nature of the device, but the device seems to lend itself to full-time use due to the complexity and fullness of the design. Being hands free was not part of the design goal of this device.

List of References
The current state of the art for technology with these possible design goals involves Electromyography (EMG). There are two papers detailing similar devices that employ Electromyography (EMG) to perform pattern recognition of gestures in combination with an inertial measurement unit. [3][4] Often abbreviated IMU, an inertial measurement unit is a sensor which typically combines various other sensors such as accelerometers, gyroscopes, and magnetometers to provide measurements related to the inertial frame of the sensors involved. The use of EMG means the sensors are placed on the forearm, leaving the user's hand completely unobstructed.
In fact, it may be possible to be used by hand amputees. The first paper, "A Novel HCI based on EMG and IMU" [3] is the earliest example showcasing such a device. The paper is unclear on many aspects of the design but essentially an accelerometer is used to move the cursor while the EMG system is used as a gesture system to trigger events such as clicking. The second paper, "Mouse HCI through combined EMG and IMU" by Forbes [4], which showcases a similar device served as a starting point for this project. This design aimed to combine the inputs of the EMG and IMU in a way that would improve the performance of the device.
To illustrate, movement in the upward direction would be ignored unless the EMG pattern recognition system indicated the user was performing a gesture of the wrist rotating upwards.
In addition to these devices, a recent commercial device called Myo [5] has become available that utilizes the same kind of EMG and IMU design. However the device is not being marketed and sold as a mouse replacement, but as a gesture input platform with a programming interface available for developers to add support for the device. As such the device is only as useful if there is an available application for what the user wants. It appears that there is no standard pointing application as a separate group has published a paper detailing their design for a pointing interface, called Myopoint [6]. A key point is that in comparison to Forbes they use a separate click and pointing interface for the most part. However they do detail three filtering or correcting techniques they used to deal with false positives from the EMG system and IMU system that take advantage of inter system design, but not to the extent of Forbes.
In addition to previous devices it became apparent that information regarding muscle physiology might be useful, specifically regarding Physiological Tremor. As the device is attached to the user's arm which will move independent of friction, the effect of tremors become much more important. While there are specific kinds of tremors associated with disorders such as Parkinson's, Physiological Tremor is a naturally occurring tremor that everyone exhibits. Its effect can be magnified and easily observed by placing a sheet of paper on one's hand. Fortunately there is much available literature on the subject, varying from studies attempting to design tremor-canceling devices for delicate surgery [7] to various studies investigating its causes [8].

Analysis of Literature
As previously mentioned the Forbes paper [4] served as the starting point for the project. A different IMU and the same but new Trigno wireless EMG system [9] were obtained in order to try to replicate the same results. Difficulties with the EMG pattern recognition system became obvious very quickly. Caution should be taken when reviewing pattern recognition papers as many will report a high accuracy, which may seem to be excellent. However, in practice a high accuracy such as 99%, combined with an update rate of 100Hz will mean an average of one false classification per second. In addition, these accuracies are often the crossvalidated training accuracy obtained from a set of data recorded in a short period of time. If the underlying random processes, perhaps such as muscle fatigue, are not stationary over the long term, a model obtained with that short-time segment of data could start to fail spectacularly later. In addition, the training data itself is not a valid representation of the full feature space that would be observed during actual use. Even with the Myo system [5], which is a proprietary system promising high accuracy, false positives can be present. Specifically, the Myopoint paper details: "Moving the arm quickly or extending it very far can activate forearm muscles leading to false-positive hand postures recognized by the EMG system" [6].
Personal experimentation with the EMG system would often obtain a fairly high training accuracy only for the system start to fail later. Majority vote and other such averaging systems were used but the reality is that they do nothing to prevent errors if the random process changes over a long period of time, or when the features are observed outside the range of training. Furthermore, the EMG system required each wireless electrode to be pasted individually to the correct position on the skin to obtain the signal from a specific muscle. Variations in the exact placement of the electrode, adhesion, and other factors related the EMG signals mean that each time the electrodes are put on the system needed to be retrained.
For pattern recognition techniques, Linear Discriminant Analysis (LDA), which was used in the aforementioned paper, and Support Vector Machines (SVM) were implemented. Both algorithms resulted in the same difficulties, but SVM did provide better accuracies. Further investigation also revealed that the individual electrodes of the Trigno wireless system were leaking electrical interference into the signals as shown in Figure 1. The presence of these signals could be the cause of difficulties. Because the electrodes are extremely sensitive it is possible that slight differences in sensor orientations or positions when recording the various classes of the training data could change the signals in a way that the pattern recognition techniques would incorrectly use those differences to differentiate the classes. Eventually the idea to use small inexpensive hall effect sensors to replace the EMG system arose and focus shifted away from using this technology.
Investigating Physiological tremor led to some varying data but most studies pointed to a peak oscillation around 8-12Hz [10].  factors of Physiological tremor is resonance of the bone and muscle structure [8].
Looking at the the spectra one might hypothesize that the combinations involving better support display clearer peaks by isolating specific joint's resonances. The other effect is that combinations without support are more chaotic in the same way that a multi-segment pendulum can swing wildly. Further study towards this hypothesis is far outside the scope of this project, but it is clear that the main source of noise and disturbance to the cursor position is an inherent phenomenon of the human body and not the IMU itself.

EMG and IMU precursor
The design process began by first attempting to develop a device similar to the one from "Mouse HCI Through Combined EMG and IMU" [1] to be used as a baseline. The EMG system [2] used was the same, but the system was interfaced differently. In the previous design the analog output mode of the EMG system was used and fed to an analog to digital converter system connected to the computer running the main system. Instead, the digital output mode was used, which transfers directly to a computer digitally over USB. The overall process is as following: a recording program was used to save a portion of EMG data to a series of files corresponding to each class. These files were then given to a training program which would output the final model file for the live program to use for classification. Cross validation was used for training both LDA and SVM. For SVM the radial basis function kernel was chosen and it was implemented using a third-party library. For both, majority vote was used to filter the resulting classifications.

IMU processing
The IMU was interfaced with by using the USB serial communications library written in C++ provided by Vectornav [3]. The Quaternion output was chosen for obtaining the orientation of the IMU as it was the simplest output type which would allow custom taring to be done in software. Each incoming quaternion sample is converted to a rotation matrix which represents the rotation as a linear transformation and used for further processing. The columns of the matrix correspond to the forward direction, horizontal sideways towards the user's right, and downwards. Each sample matrix is first adjusted by the taring system detailed in the next section.

Taring
The taring system is used to allow setting a point of reference that further measurements are made from. This is similar to zeroing a scale before taking a measurement. To create a taring system a zeroing rotation matrix is calculated.
This matrix is the inverse of the yaw rotation when the system is tared. Multiplying this matrix against each incoming matrix makes the output relative to the original matrix in terms of the yaw orientation. The pitch and roll orientations are unaffected as their zero points do not change. The algorithm to obtain this matrix is to essentially flatten first column of the tare matrix, and to set the third column to straight up. To illustrate this, consider the following example rotation matrix which represents an arbitrary rotation around the third axis: In this example, the third coefficient of the first column is zero, but this is not the case with full rotational freedom. However, one can simply zero out that value to obtain a vector in the correct orientation. This has the same effect of simply projecting the forwards direction of the IMU into the horizontal plane.
Since rotation matrices should be orthonormal, the resulting vector will need to be normalized to ensure this. Filling out the rest of the new rotation matrix is easy as the second column is simply a rearrangement and negation of the first column.
Finally, the whole matrix needs to be inverted, which is easily done with a simple negation of coefficients corresponding to the odd sine function. The resulting matrix, given the normalized vector x 1 x 2 0 T , the final matrix is as follows:

Translating 3-Dimensional data to Screen Coordinates
For the IMU rotation to be used to drive cursor movement a scheme for translating rotation to (X,Y) screen coordinates is needed. The simplest scheme is needed is to simply get the tangent of the pitch. This is just the ratio of up to horizontal of the first column of the rotation matrix. If we represent that column as x y z T then the mapping ends up as follows: The constant d would be the distance to the wall, but really just serves as a scaling coefficient. However, previously the scheme was simply Y = pitch, with pitch in degrees. It would be useful for the two schemes to match in scaling, at least when close to zero pitch. This led to the following equation: Setting pitch to zero doesn't work quite as well as it simply it is the algebraic equivalent of throwing the equation into a black hole. However setting pitch to one, which is quite close since pitch is in degrees, works quite easily: As a quick observation, one can note that at the value of d is equal to Y when the pitch is 45 degrees.
Finally, one minor nuance to dealing with the X value is needed. Specifically it is calculated nominally as following: Ultimately the problem with this is that the arctan function is periodic when it should not be. Instead of this, the previous yaw value is stored and the difference to the new yaw value is added using the following equation:

Using Roll
While yaw and pitch rotations primarily drive cursor movement, the sensor also can provide measurements of roll rotations. This was found to be highly useful due to a clear difference between the roll orientation from a natural pointing pose to the typing pose. This meant that input could be temporarily disabled automatically when the user goes to type, and reactivated as soon as the user returns to pointing. This feature is quite similar in use to palm-recognizing features designed to disable the trackpad on a laptop when the user is typing, but is much more reliable. In addition, rolling the forearm over, as with wrist supination, is unnatural while pointing, and can be used to quickly and reliably indicate something to the system. It was decided to use this action to switch the device into sleep mode. This was useful as it also performed well as an emergency-stop in case the device started to produce undesired input. It was also decided that the condition of the pitch angle being extremely high or low, beyond the range of comfort, would activate the same effect.
Calculating the current roll from the current rotation matrix is more complex than calculating yaw. Representing the rotation matrix as x y z , the important vectors are x and either y or z. As x represents the direction of pointing, the rotation of y and z around it corresponds to the roll angle. In this case the vector z is chosen for the calculation. First, the goal is to remove the pitch rotation from the vectors. We can zero out part of the x vector and renormalize to make the next step simpler.
Next, use GramSchmidt orthogonalization to remove pitch from the z vector using the x vector. In this case normalization is not needed.
Now, set up two normalized two-element vectors to be used in the final step: Finally, the angle can be calculated via arctan using the adjusted z vector. The adjustments allow the x and y values needed for the arctan2 function to be obtained simply. The determinant calculation simply provides a value of -1 or 1, giving the needed sign missing due to the always positive square root calculation.

Hall Effect Sensor Processing
Developing the Hall Effect Sensor portion of the device turned out to be straightforwards. The main issue was that there is some sort of periodic interference somewhere in the ADC pathway. The design plan was to create a simple detector to detect whether the magnet was present near any of the hall effect sensors or not. The final design ended up being essentially that but with a different threshold for activating and deactivating, as other than the periodic interference there were no other complications.

Interference
The interference observed is a periodic signal with a long component and a discrepancy during the short components. This can be seen in figure 3. With this modification the effect is reduced, and afforded a much tighter set of thresholds and higher sensitivity was achieved.

Detector design
As the interference signal was much larger than any apparent sensor noise, building a noise model for a detector simply involved recording data from the sensors. Thresholds were then set using the maximum or minimum values recorded, depending on which was relevant. Doing so is essentially setting the desired probability of false alarm to zero, which would normally adversely affect detection performance. However, since the noise signal is pretty well behaved, the thresholds stay tight enough so that this works absolutely fine. In addition it was decided to use a different threshold depending on the previous state of the detector. That is, when there is no magnet present, the threshold used for the next sample is a little higher, requiring a stronger signal to be a detection. This is primarily in case the signal drifts near the normal threshold, where oscillations due to noise could cause the signal to go above and below repeatedly. For this kind of system a detector that changes state rapidly is undesirable as it will trigger unintended mouse clicks.

Managing Physiological Tremor
Ultimately the performance of the pointing system comes down to two things: being able to move the cursor finely while maintaining responsiveness, and the ability to hold the cursor still. The main culprit affecting performance in both these cases turns out to be a phenomenon known as Physiological Tremor. It is typically characterized by peak oscillation around a specific frequency, typically around 7 to 11Hz. While there are many types of tremors with various clinical significance such as Parkinsonian tremor, Physiological tremors are present in all humans. Furthermore, there is evidence to support the theory that one of the main mechanisms behind it is physical resonances of the muscular and bone structures of the human body. In this way, the apparent oscillation is not a result of muscle firing at a specific frequency but rather the muscular and bone structure resonating at that frequency. If it was the case that muscle firing contributed to causing these oscillations one might expect to be able to observe the muscle firing in order to cancel out the disturbances. Instead, the muscular and bone system under question probably acts more along the lines of a car's radio antenna vibrating wildly when driving down the highway. Additionally there is evidence that muscle contraction, as during dynamic movement, changes the frequency of oscillation [5]. Altogether this is a daunting source of noise and disturbance to the pointing system.
In the case of allowing fine control while maintaining responsiveness, there is a trade off in terms of an arbitrary scaling factor between the actual user's movement and the corresponding number of pixels that the cursor moves. Being able to adjust this scaling factor is a standard feature for any computer and more or less exists due to differences in screen resolutions and the user's preference. For example, a way of setting this scaling value would be to determine how far the mouse must move to traverse the entire screen. The distance needed should fit somewhat comfortably within the space the user has to move in. On the other hand, one could set the scaling around being able to do fine pointing accurately by reducing the sensitivity.
This reduces the effect of disturbances and allows the user to make much larger movements which can be done more accurately. However, doing so can reduce the responsiveness of the cursor, meaning the user might have to pick up the mouse in order to cross the entire screen. One feature that attempts to strike a balance is mouse acceleration, where stronger motion is amplified, allowing the user to shoot the mouse across the screen but still maintaining fine control for weaker motion.
However, the nature of the acceleration can be somewhat arbitrary to the user and ultimately is up to the user's preference. Applying such a response curve to the motion of the cursor for this design as a default should not be done without due process and reason.
The second case, being able to hold the cursor still, is easily achieved by normal mice due to friction. However, for this design the reality is that the user cannot hold the cursor perfectly still on their own. The lack of friction means even the slightest disturbance or twitch changes IMU's velocity instead of being canceled out. The ideal design for this would be to emulate the effect of friction using the input from the frictionless system. This is challenging, because while one could simulate friction acting on the raw cursor movements, that would cause the cursor to stop while the user is still moving the IMU. This disconnect is jarring to the user and in practice just following the raw cursor movement during large IMU movement works fine. The trick is to only affect the cursor movement under weaker movements, where friction might stop the cursor from movement.

Precursor Design
The first attempt at a solution to this challenge was essentially a sensitivity curve. The curve mapped the raw delta in cursor position in the x and y directions independently to a new value of reduced magnitude. This is essentially taking the idea of mouse acceleration but going the other direction and dampening smaller movements. The curve can be described by the following function: The parameter a is half the width of the 'deadzone' of this function, where the function equals zero due to the unit step function on the right side of the equation.
This is an important feature of the curve as its intent is to allow the cursor to be held still if the user can keep their movements within the deadzone. The parameter b controls the drop off speed and shape of the function. It was set to one, because at that value the second derivative is zero at the edge of the deadzone. A example of this plot can be found later in figure 5 in the first column. Since the parameters are not defined, suitable values would have to be determined in testing. However a problem was quickly apparent. While it was possible to set the deadzone to be wide enough that the cursor could be held still, fine cursor control became impossible.
If the user wishes to make a small adjustment, they must move quick enough to surpass the deadzone, but only just over. This region of the curve has a high slope, which means that the user has to be much more accurate than normal. Errors in movement are amplified by a value equal to the slope. Making small adjustments accurately is challenging enough without this added difficulty.

Main Design Part One: Gaussian Based Sensitivity Curve
The next attempt at a solution involved more foresight. First, a data record was taken of the cursor's movement while the user attempted to simply hold the The value of the two probabilities can be restated with the CDF using the Which using the techniques for a Bayesian Means Squared Error estimator detailed in Kay [6] finally leads to an equation somewhat similar to the precursor design: A comparison of this sensitivity function, the precursor sensitivity function and the sensitivity function used by the Myopoint design [7] is provided in figure 5.
The important thing to note is that this sensitivity curve stands out because it is based on an actual measured distribution. The sensitivity curve attempts to reduce the overall error by reducing the movement based on how likely the observed movements was from intentional or unintentional movement by the user. This estimator provides an excellent solution to the first problem of obtaining finer pointer control while maintaining responsiveness. However, this estimator does not solve the second problem as there is no deadzone or other effect that would allow for the cursor to be held still.

Main Design Part Two: Simulated Friction
The second problem of being able to hold the cursor is still not solved and poses a more difficult challenge. Any reasonable solution is going to include some form of deadzone that causes the cursor to stop moving. The simple solution, just a flat deadzone, is not acceptable as it affects fine pointing control. Any design which applies a deadzone on all input is not a very good design as it makes it impossible to achieve small input. A modification would be to use a double threshold switching system with a high 'move' threshold and a low 'stick' threshold. Additionally while the sensitivity curve approach lends itself to processing on a sample-by-sample basis, a thresholding system could benefit from using more than one sample at a time. However, the analysis is more difficult as this introduces correlation between samples. Because the IMU is a physical object it must accelerate over time, resulting in each sample being fairly close to other recent samples. In addition there is the effect of physiological tremor which is typically characterized by oscillation.
In an attempt to understand the effects that this correlation might have, a set of data was recorded and analyzed to produce a series of power spectral densities ( figure 6). Five combinations were recorded by simply recording the raw cursor signals using the working prototype system, with the IMU mounted at the back of the palm or on the wrist. For the case of the back of the palm, recordings were made with the entire arm unsupported, supported at the elbow, and supported midway on the forearm, isolating different joints from movement. For the wrist, recordings were made with and without the elbow support, leaving out forearm support. In addition a recording was made with the IMU simply sitting on the desk and not mounted on the arm.
As mentioned in chapter 2, the case of the palm mount with forearm supports shows the clearest peak. All of the mounted cases show significantly more energy than the unmounted recording, which indicates that physiological tremor is the main source of disturbance and not sensor noise on the part of the IMU.
Unfortunately the only case that shows a clear spectrum where a filter might be of use is the palm with forearm support, which was included to compare with other studies of physiological tremor, not because it would be a viable way to use this device. Mounting the IMU on the back of the palm and supporting the wrist would isolate the wrist as the joint to control the cursor. This would not be viable as the range of motion of the wrist is limited, compared to the elbow. A useful comparison however, is to compare the unsupported cases, to determine if mounting the IMU on the wrist or elbow affects the noise levels. In this comparison the wrist mount overall has slightly less noise. The wrist mount had been chosen originally on the intuition that it would be more stable, so no changes were made. The spectrum for this combination is shown in figure 7.
Unlike the setup of the IMU mounted on the palm with forearm support, the chosen setup has a smooth spectrum in comparison. It was decided to assume white Gaussian noise as it would simplify the problem and allow for some sort of solution to be devised. A possible design is to create a detector using a linear model of the cursor following a constant velocity. The model would be specified  as follows: With this linear model a generalized likelihood ratio test (GLRT) [6] can be easily Note that while one would normally include position parameters somewhere in a model of linear movement, h is set up to have an average of zero. This means that any constant vector will be orthogonal to h, rendering any positional offset irrelevant to the calculation. There are two parameters still left unassigned however; the window length N, and the relevant thresholds to be used. In addition, the square root of the final test statistic is not necessary for the threshold test but is included so that the test statistic is the actual speed estimate. To this end, data segments were obtained featuring a strong, moderate, and weak single movement, to complement the usual 'noise' recordings. Using this data different values of N could be compared, and thresholds determined. The results of three window lengths are featured in Figures 8, 9, 10 and 11. For the sake of plotting, the strong movement signal has been left out as they were off scale of the charts even when plotted at log scale.
A good window length to choose is one which strikes a good balance between smoothing the signals and being large enough to cause noticeable response delay.
Of course, if the onset of a movement is strong enough the delay is shortened because the detector could be triggered before the new movement fills the window.
Therefore the delay should cause the most impact for weak movements. With a 100Hz sample rate, a window length of 25 corresponds to a quarter second delay in the worst case. This value is no accident as it is roughly the normal human reaction time to visual stimulus [8].
Before deciding on the thresholds to use, there is an interesting observation to make: there is a characteristic difference between the noise signal and weak movement signal. While the weak movement signal remains flat, the noise signal rapidly dips sharply. This is likely due to how the test statistic turns the two If a two threshold switching system is used, the stick threshold, such as noted by the red horizontal line in figure 10, the stick threshold can be set quite low.
Although the signal is not constantly below the threshold, the rapid dipping means that the signal will pass under the threshold very often. Since this is a switching system, each of these dips will switch the system's state to stop the cursor from moving. As long as the threshold is not set too low, any potential delay may not be noticeable. Another consideration is that the user may cause a dip that will stick the cursor immediately at the end of their movement by actively arresting their own movement. The threshold start allowing movement again was simply set to the upper limits of the noise signal. This threshold should be set low to maintain responsiveness, but could be left to the user to adjust if they prefer. However, if set low enough, the user can start the cursor low enough in the sensitivity curve to still be able to make fine adjustments.  Testing the prototype was done by simply trying to use the device to perform the normal functions of a cursor. Early designs proved frustrating as the cursor either remained unresponsive or could not be held still in order to successfully perform a click. However, the Hall effect design, once the interference was reduced proved to be reliable. Given the inexpensive nature of the sensors this was an effective alternative to utilizing EMG pattern recognition to achieve clicking control in terms of ease of use, reliability, and cost. The Hall effect design still allows the device to be hands-free by avoiding placing functionality such as buttons obtrusively inside the user's palm.

List of References
After incorporating the two main features of the IMU processing, the sensitivity curve and friction simulation, the device saw remarkable improvement.
While the sensitivity curve was developed early, it alone did not solve the problem of the cursor constantly dancing around. Clicking properly was only possible by quickly 'tapping' and only in cases where dragging at the cursor's location was not possible. However, once the friction simulation was added, all these problems disappeared. The cursor sticking effect felt familiar and intuitive, given extensive experience with normal mice. Though the IMU may be moving, the actual muscle movement and cursor response feels natural.

Design Improvements
Though the prototype was successful as a proof of concept, it is not quite a complete device. There are some obvious improvements that would be included.
In addition to the obvious wireless interface and battery power that would allow for a true untethered device, scrolling and additional button functionality was not developed for the prototype. However, extra button functionality could be added by including the ring finger, and by making use of the duality of magnetism. Each Hall effect sensor has the possibility of detecting a positive or negative magnetic field across its detector. The prototype simply detects any deviation from normal levels, not distinguishing between the different field directions. If the field direction was considered the number of touch spots could be doubled, allowing for additional mouse buttons. A total of six should be theoretically possible, allowing for three additional mouse buttons corresponding to middle click, and backward and forward buttons. The sixth combination could be used to switch the device into 'scroll' mode, where the IMU's movement is used to scroll instead of moving the cursor, similar to the two-finger scroll common on touchpads.
Another possible design improvement would be to make use of Haptic technology. As such for the prototype the only indication that the user is clicking with the Hall effect sensors is that there is a corresponding response on the screen. This is not a substantial problem as the the system is very reliable and consistent, but it is possible that the use of some form of Haptic technique could improve the design. The touch response of pressing a button or clicking the mouse provides the user with feedback to confirm that their input is being accepted. However determining the most effective type device to use and exactly what kind of feedback to provide could be an entirely new study.
Besides these improvements, this device has additional potential that would require development of new interface designs. Because the IMU is not restricted to two-dimensional space, there is the potential to make use of a full six degrees of freedom movement between x, y, and z acceleration and yaw, pitch and roll rotations. For instance, to navigate a 3D virtual environment, one could 'grab' with the mouse and then move the IMU in the desired direction. Or when examining a 3D object, the full 3D rotation could be used to rotate the object on screen.
In addition to the obvious sensor advantages, the untethered and hands- Additional applications may also be developed in which the ability to use two hands would be useful. In the end, this type of device stands to provide unique opportunities for modern human-computer interactions.

Source code for main application
In this Appendix source code for a C++ application is provided. This application is run on the user's computer. It provides a visual display of the hall effect sensor data, handles communicating with the microprocessor, and also handles actually sending input from the device to the computer system.

A.1 Hall effect processing object
This object abstracts the hall effect processing, automatically creating a data collection thread and handling functions such as checking clicking and drawing the signals.