Smartwatch-driven multisensory recorder: Design and testing of a smartwatch-based framework to support psychiatric disorders

The advancement of technologies such as Bluetooth Low Energy (BLE) and the availability of low cost sensors has allowed the Internet-of-Things (IoT) industry to develop rapidly. The Android application described in this paper takes advantage of the opportunities provided by these technologies. This application can simultaneously collect data from all sensors available on the Microsoft Band 2. It then stores this data to be later viewed and analyzed. This method of data collection could help in many cases, including psychiatric disorders. For example, our app is aimed at monitoring physiological data (heart rate and galvanic skin response) and ambient environment data (sound and lighting conditions) of trauma exposed individuals with posttraumatic stress disorder (PTSD). The preliminary data collection study on healthy individuals demonstrate that our framework efficiently handles the continuous recording of multiple sensors.


I. INTRODUCTION
Today, an overwhelming number of citizens face mental health challenges.For example, post traumatic stress disorder (PTSD) that is a kind of psychiatric disorder manifesting various emotional and behavioral symptoms such as anxiety, mood swings, depression, anger, impulsivity, and many other issues is prevalent in adult Americans at the rate of 6.8% [1].These individuals with PTSD receive various behavioral therapies.It is very difficult to personalize the therapy based on the manifested symptoms.Therefore, it is very important for clinicians to monitor their patients with PTSD when they are at home.Our paper presents a smartwatch-based data collection framework, called "Android Electronically Activated Recorder (anEAR).
The anEAR framework uses the multi-dimensional sensors built into the Microsoft Band 2, alongside a Bluetooth capable Android device.The framework includes a custom application that uses the smartwatch from which the Android device receives sensor data.anEAR is able to simultaneously collect sensor data from several sensors while also periodically recording audio using the device's microphone.This allows users to view various individual's biometrics while also being able to pinpoint the cause of irregularities by matching the audio recordings to the sensor data.

A. Android App Development
Android is an open-source operating system based on the Linux kernel and sponsored by Google.It runs on many different systems including smartphones, tablets, smartwatches, and more.Android's Software Development Kit (SDK) provides many different Application Programming Interfaces (APIs) that allow developers to take full advantage of all of the technologies available on the Android device.Due to its ability to run on so many diverse devices along with all of the APIs available, Android is the ideal operating system upon which to build the anEAR framework.

B. Smartwatch -Microsoft Band 2
The Microsoft Band 2 is an activity tracker developed by Microsoft which features continuous streaming of heart rate data along with ten other sensors.The eleven sensors are: optical heart rate sensor, three axis accelerometer, gyrometer, Global Positioning System (GPS), ambient light sensor, skin temperature, ultra-violent sensor, capacitive sensor, Galvonic Skin Response (GRS), microphone, and barometer.A description of each sensor can be seen in Table 1.The Band can connect to any iPhone or smartphone running Android or Windows operating system.The band connects using its own application which must be installed on the device at all times.Furthermore, the band also provides its own SDK for iOS, Android, and Windows that allows developers to access the band's sensors at any time and to display notifications on the band's screen.package provided by the band was the driving factor in our decision to use it in the anEAR framework.

III. APP DEVELOPMENT & PROGRAMMING
The Android application for anEAR was designed to have a straight-forward User Interface (UI) while handling as many errors as possible without stopping the entire application.In order to do this, the application follows several fundamental design guidelines created specifically for anEAR.

A. anEAR Activities
In Android, Activities are a main component of any application since they define the UI and handle any events corresponding to the UI such as button clicks.In anEAR, every Activity provides a way to graphically show messages to the User and contains static methods that handle all responsibilities related to the Activity.For example, the Settings Activity provides static methods to access and manipulate the settings that anEAR uses.This allows for each Activity to be completely independent of each other while providing a layer of abstraction between the client, in this case the Activity accessing the settings, and the provider, in this case the Settings Activity.The reason for this is to make further development on anEAR as efficient as possible and this allows new Activities to be easily implemented and existing Activities to be easily updated.For example, if the settings on anEAR needed to be moved from residing in a file to being stored in a database, only the Settings Activity would have to be modified and the rest of the Activities would not even be aware of the change.anEAR consists of four Activities: Settings Activity, Lock Screen Activity, Blackout Settings Activity, and Main Activity.Main Activity is the first activity that is launched when the User starts the application.First, this Activity checks that the device has all the correct permissions and contains all the necessary hardware.If not, it displays the appropriate error message to the User.Next, it determines whether or not the application has a pattern lock set and starts the Lock Screen Activity, passing this information to it.anEAR is meant to always be locked with a pattern lock, so if Main Activity does not find a pattern lock saved, it prompts the User to create one by starting the Lock Screen Activity to create a pattern lock.
Lock Screen Activity is responsible for creating and changing the pattern lock for anEAR as well as making the User enter the correct pattern lock to continue in the application.anEAR uses a third party library [3], to display the pattern lock as well as providing information about it, such as what the pattern is.
Settings Activity is responsible for displaying all of the current settings for anEAR as well as providing the User with a way to manipulate each setting.The User can edit the duration of an audio or sensor recording, the delay between audio or sensor recordings, the sensors to be recorded, the sensors to trigger automatic recordings, and the name of the root folder where all of the data is saved.
Blackout Settings Activity is responsible for creating, updating, and deleting blackout periods.Blackout periods are periods of time where anEAR will not record and periodic audio recordings.This does not affect sensor recordings, which are toggled in a way that will be explained later.anEAR allows the User to specify a start time, an end time, and whether this should take effect on the weekends or weekdays.The values are then encoded into binary strings and saved into a SQLite database on the device.

B. anEAR Services
Services are another main component in Android.Services are very similar to Activities except they do not have any UI and can be run on a separate thread than the application's main, or UI, thread.In anEAR, Services are used to connect and collect sensor data from the Microsoft Band as well as saving this data on the device.anEAR only connects to the Band, and therefore only collects data, via Collection Services.These Services are made specifically for anEAR collecting data from the Band.They automatically connect to the Band when they are created, start a timer once connected, and disconnect from the Band once the timer is finished.The duration of the timer is based on the amount of time the User specifies for recording sensor data.By using this method to connect to the Band, cases of unnecessary or unclosed connections are significantly decreased which significantly increases battery life of both the device and the Band.Each sensor that is to be streamed from the device creates a new Collection Service.By using individual Services for each sensor, anEAR is able to gracefully handle any error related to one sensor while streaming all other sensors without issue.
anEAR uses one main Service, named Record Manager Service, to coordinate all Collection Services.Record Manager Service is triggered anytime either a sensor or audio recording session is scheduled to begin or should be canceled.The Service then checks that it is able to and allowed to carry out whatever action it is supposed to carry out and executes it out if it is.For example, if Record Manager Service is started with the intention of beginning one of the periodic audio recordings, it will check that it is not in a blackout period and then start the Audio Collection Service.a size of around 1.90 kilobytes.By using the CSV file format, the sensor data that is stored by anEAR can be very easily viewed but also gives the User freedom to use the data however he or she chooses.anEAR stores the audio recordings in a Waveform Audio File (WAV) format.WAV files contain a 44 byte header followed by a body of uncompressed audio data.Although using the WAV format sacrifices some memory space due to the fact that the audio is not compressed, it allows for a higher quality sound which could be very important for many of anEAR's various use-cases.
In a futher attempt to reduce errors, anEAR also contains a CSV file named Audio Record Log.This file contains the start time and date, end time and date, along with other useful information about every audio recording session.This allows Users to go back and ensure that anEAR worked successfully throughout the time it was used and also to see if any audio recordings failed.

V. CONCLUSIONS
We developed a smartwatch-based framework called anEAR to record continous physiological and environmental sensor data, providing insight of how the wearer is interacting in social and personal contexts.anEAR is particularly aimed at tracking emotional arousals in daily lives of individuals who develop post traumatic stress disorder (PTSD).The paper presents the architecture of the system.We successfully demonstration of anEAR in operation to record the multisensor data from smartwatch.Our future plans include the deployment of this app in study participants with PTSD.

Figure 2 :
Figure 2: A workflow diagram describing the process to begin sensor recordings.

Figure 4 :
Figure 4: Diagram demonstrating how Record Manager Service uses separate Collection Services for each sensor, allowing for each sensor and its errors to be handled separately.IV.EXPRIMENTS AND RESULTSanEAR produces Comma Separated Values (CSV) files for each sensor's recording session.This allows for Users to view to data in any variety of ways including spreadsheet tools such as Microsoft Excel or a computing environment such as Matlab.These files take up very little memory, for example a one minute recording of heart rate data creates a CSV file with

Figure 3 :
Figure 3: App screenshots of AnEar.Pattern lock restricts the user to change the settings.Settings are for clinicians to provide therapies who can select sensors and their sampling periods.Sensor data are logged onto the phone memory.

Figure 5 :
Figure 5: Time series graph created using heart rate data recorded by anEAR using the Microsoft Band 2's heart rate sensor.

Figure 6 :
Figure 6: Time series graph created using GSR data recorded by anEAR using the Microsoft Band 2's GSR sensor.

Figure 7 :
Figure 7: Time series graph created using RR Interval data recorded by anEAR using the Microsoft Band 2's RR Interval sensor.

Figure 8 :
Figure 8: Time series graph created using the skin temperature data recorded by anEAR using the Microsoft Band 2's skin temperature sensor.

Table 1 : List of built-in sensors of Microsoft Band 2.0 [2]
centimeters, current speed in centimeters per second (cm/s), current pace in milliseconds per meter (ms/m), and the current pedometer mode (such as walking or running) 1 Hz Heart Rate Provides the number of beats per minute; also indicates if the heart rate sensor is fully locked on to the wearer's heart rate. 1 Hz Pedometer Provides the total number of steps the wearer has taken since the Band was last factory-reset. 1 Hz UV Provides the current ultraviolet radiation exposure intensity.1 Hz Band Contact Provides the current state of the Band as being worn/not worn.0.2/5 Hz RR Interval Provides the interval in seconds between the last two continuous heart beats.Value Change Ambient Light Provides the current light intensity (illuminance) in lux (Lumes per sq.meter).