Smart Fog Computing Interface for Healthcare Domain Applications

Technology is advancing tremendously in this day and age where more and more devices are being deployed in the market of digital devices; so much so that the market is being oversaturated with many different kinds of digital devices to simplify our everyday needs in life. This advancement can have negative consequences, however, there may also be benefits that stem from this technological advancementespecially in healthcare. One possible benefit is to utilize the device’s capabilities to give real time feedback to people with medical diagnoses. Furthermore, that device could be connected to a stream of other devices to have a greater impact towards diagnosing and treating patients’ health concerns. This platform is known as the Internet of Things (I.o.T.). The I.o.T. is a network of wireless physical devices that are capable of communicating with one via embedded systems, microcontroller devices coupled with sensors, or any other device that is capable of acquiring data. In this thesis, we investigated the speech of people with Parkinson's disease (PWPD) by applying the I.o.T. computing interface architecture to provide a better solution for diagnosis involving healthcare providers, patients, and administrators. First, we utilized a localized or mobile approach to acquire audio data from a patient with Parkinson’s disease. We investigated the different capabilities of wireless connected devices such as a micro-controller to collect audio data from PWPD, transfer the data securely over the network, process the data for specific means of data extraction and diagnosis, save sensitive audio data extraction, and provided a structural means of data interaction for the user of the processed data to be visualized in a Graphical User Interface (GUI) setting. Our novel means of data collection, transfer, and processing which deploys a micro-controller allows for a localized network usage that minimizes security risks and improves the mobile device’s processing which in turn saves on power consumption, and allows for quick extracted information to be saved on the cloud that decreases the risk for privacy issues. Our method also allows for doctor and patient interactions where a doctor has the ability to evaluate the user’s (who also happens to be the patient) progress without having to physically meet the user. We expect that our methodology and implementation of the I.o.T. architecture will improve treatment of PWPD.

Fog computing is defined as a distributed computing paradigm that fundamentally functions as a middleman between the sensors and/or actuators and the cloud [3]. Fog computing generates three tier architecture (clients  fog  cloud). The key benefits of adopting fog computing over traditional cloud computing include; reduction in network traffic [4] and data storage on the cloud [5]; low response latency in I.o.T.
applications demanding real-time actuations such as smart traffic lights [6].
Fog computing is increasingly penetrating the area of healthcare, especially to improve tele-health and telemedicine infrastructure that promise to cope with the rising healthcare needs in elderly population and individuals with chronic conditions around the world. For example, fog computing upgrades the standards of body sensor networks for medical signal processing, energy-efficient computing, and privacy and security [7]. in the healthcare domain was met. We were motivated to construct a simplified but robust layered system where a healthcare challenge that requires data collection and diagnostics could be utilized and supported in the structure of our study and research in this thesis.
The healthcare challenge that was addressed in this study was a solution for speech deficits in PWPD. In 2010, the National Institute of Deafness and Other Communication Disorders (NIDCD) reported that 7.5 million individuals in the USA alone have speech disorders [10]. Parkinson's disease (PD) is also categorized as one of such communicative disorders that not only affects motor activities but also impairs the speech and voice functions in over 90% of patients [11]. Patients with PD experience a combination of speech impairments in various ways including: reduced vocal loudness; a breathy or harsh voice quality; imprecise consonants and distorted vowels; and reduced voice pitch (fundamental frequency) variation. Hence, patients with PD go to speech language pathologists (SLPs), who diagnose their speech impairments and then give them personalized speech treatments targeted to restore or improve the impaired speech functions. The challenge to treatment occurs when the patients are given speech exercises as homework on their mobile devices by the SLP. PWPD have a progressive neurological disorder that involves motor and non-motor symptoms that can interfere with the successful completion of home speech exercises that are vital to the speech treatment regime. Finally, we seek tele-health technologies to monitor the progression of their patients remotely.

OBJECTIVE (PROBLEM STATEMENT) AND INQUIRIES
We utilized the architecture of the I.o.T. system, including Fog Computing, which provides an interface where multiple devices are used for interaction between numerous users involved. This technology allowed for a configurable platform where different users of our system could interact with one another in multiple instances. The work included created a mobile application on the Android platform so the administrator, doctor, and patient could interact with one another to accomplish better diagnosis of PWPD. We called our work, F.I.T: A Fog-driven Internet of things interface for smart watch-based speech treatments.
F.I.T started with the goal of analyzing recorded speech from patients that we gathered on the fog computer. We wanted to implement a fog interface to process and analyze information needed from the audio files quickly. Then we advanced our work further so we could include the interconnection of physical devices, software, and people who could be helped because of it.
The goal of the study was to develop an I.o.T based system where a gateway (Fog, Edge) computer could participate in layered systems to interface with the cloud and different kind users to provide a task specific solution. Furthermore, we wanted to develop a system that was user intuitive for most of the process after the initial setup.
To complete the aforementioned goal, the following research questions were addressed: 1. What was the main objective of the Fog computer in the I.o.T. based system?
2. What other connected devices, components, and software were going to take part in the research?
3. How would we be able to design software in an administrative structure to assist our users?
4. How were we going to design our system so that it is configurable?
5. How were we going to design our system so the ease of use could be as simple so that setup was as minimal as possible?

REVIEW OF THE LITERATURE
The structure of this chapter will define I.o.T. and its application to the healthcare domain. In the first section we will explain the I.o.T with some advantages of it and its architecture design setup. The following sections will seek to explain the required modules to design and implement an I.o.T system. The intelligence of I.o.T. is its ability to use a variety of physical devices in an intelligent fashion to solve and simplify a societal problem. This intelligence has been taken notice by many educational and professional institutions. I.o.T is offering a great market for many. It is now being predicted that the physical devices that will make I.o.T based systems will estimate about 212 billion globally by the end of 2020 [12].
Furthermore, according to McKinsey Global Institute, the number of the devices that communicate with another (M2M: machine to machine) has increased by 300% in the last five years [13]. Furthermore, I.o.T. is beginning to influence the market economically at such a rapid pace in many different fields. In figure 1 we see an example of this. [14] [13].

FOG COMPUTING
According to Cisco, Fog computing are physical devices which are nodes employed anywhere with the connectivity to the network and has the following operations: computing, storage, and network connectivity [15]. Because of these attributes, there are multiple reasons that fog computing is becoming a factor in today's market. Following are some are the reasons why the Fog is beneficial and below is a

Cost:
Large networks are constantly being buffered with tons of data that are circulating, bottlenecks are more susceptible to occur as those same resources are being utilized. In the case of Fog Computing, bottlenecks are less frequent to take place since there is less amounts of data that is being moved throughout the network since it is centralized. This can be less cost in terms of bandwidth transmission in the network [16].

Network Expedience:
The ability to process and analyze data closer to its originating source can significantly minimize network latency [15]. This expedited computation of the data to be executed. This is also a benefit since many services require very low latency in regards to response to an action. In areas like banking, traveling, energy, health, I.o.T.
applications are playing a huge role in effective computing with as low latency as possible [16].

Security and Privacy:
I.o.T plays a significant role in the security and privacy conversation since the vast amount of data involved is sensitive personal information that can create personal safety. In essence, Fog Computing can have its own implementation of security and privacy to prevent hackers from stealing important information just as other security procedures that is used in the information technology environment [15] [16]. In conclusion, Fog computing allows for faster processing and communication since it is closer to the source of the data, decreases the risk for data theft traveling on the network, and increases the protection of the data, minimizing any network bottlenecks if the data becomes immense.

I.o.T. ARCHITECTURE
In this section we will define the I.o.T. structured layered system that most of our work took advantage of. The following will define each of the layer specifically and the table 2 [14] to follow shows what kind of devices and/or platforms that can be used at each layer. The layers of the Architecture are the following [17]:  Hardware Layer (Fog layer) ▪ The purpose of this layer is to encompass physical objects such as temperature sensors, or any kind of sensor that collects data such as embedded computers, actuators, displays for notifications, etc.
Moreover, the purpose of this layer can be to process information, store, and or communicate the information or processed information with another interface.

 Communication Layer (Access Layer)
▪ The purpose of this layer is to set up the communication required with the other platforms that data can be managed and or transferred to.
 Software Layer (Data interface Layer) ▪ The purpose of this layer is to store the compiled information received from the hardware layer so that it organized efficiently for the application layer.

 Application Layer
▪ The purpose of this layer is to realize a form of data visualization.
This layer can take shape in many different forms for various types of applications. This layer can be specific to the nature of the application.

 Security Layer
▪ This is not a typical layer of the I.o.T. platform, although as mentioned in the above section, the implementation of security at any layer can be beneficial especially depending on the kind of data that is being collected. In the healthcare realm, it is crucial for healthcare providers to supplement their practice with technology to maximize effectiveness of treatment. The treatment that I.o.T brings to the healthcare industry and in our research is the ability to monitor the patient and diagnose a patient from a remote location. One of the ways to accomplish this is by the means of wearable devices and their ability to also connect to the network.
From versatile patient monitoring such as smart-watches, wearable devices or even mobile devices, the healthcare industry tremendously benefits from I.o.T. especially when finances are the topic of conversation. Figure 3, illustrates the potential of the healthcare application in I.o.T.
According to the Kaiser Family foundation, hospital stays are costly; the average daily cost for a patient's one night stay in the hospital was over $1,700 [18]. For most of population this can be very bearing. Remote monitoring can be beneficial in minimizing such high costs since wearable devices such as smart watches and fitness trackers are relatively inexpensive. Another cause for concern in the health care industry is the cost of drugs. According to Forbes, an average of $55 million is what it takes to develop an approved drug. Also, there are others who are making a multi-million-dollar profit for making unapproved drugs. It is a possibility that using the I.o.T. system, some of these costs may be minimized [19]. Some industries are embedding technology in drugs to keep track of them at different stages of the manufacturing and development process [20]. Other companies like Proteus Digital Health have developed pills that track the consumption if and when a patient has consumed their medication [21].
Even with all of aforementioned possibilities of the I.o.T. based systems in the healthcare field, security is still a major concern. Some systems in the I.o.T realm are being developed in such a way that when sensitive data is being collected for privacy measures, specific information about the patient is intentionally not managed in no other place rather than the Fog itself. For example, in our research, we do not allow for the storage of the patients' data to be stored on the Fog computer for more than a week. We also make sure that when the information is passed to the node or Fog computer that belongs to the patient, we encrypt the connection before the data is passed to the edge node for further analyses.
Lastly, according to I.O.T. Analytics, wearable technology and connected health technology (Tele-health) is increasing rapidly according to figure three [22]. This was part of our motivation to take on this thesis research project.   In conclusion, because of its compact size, its capability, its power consumption, and its configurability to change size, the Intel Edison was chosen to be our Fog Computer.

I.o.T. MIDDLEWARE
The purpose of the middleware is to provide means of communication between the software and hardware involved in the I.o.T. platform to perform specific machine to machine tasks. According to Digital Service Cloud, the role of the middleware is to allow and enable the interaction of the internet of things hardware and software to allow for smoother communication [25]. In the case for our research thesis, we took on the challenge to design a seamless as possible software that expert users will have as little as possible knowledge but be able to interact with the system easily.

COMMUNICATION/ACCESS LAYER
The purpose of this layer in our study was to make sure that the data that we were collecting was being able to communicate in a secure way to the Fog computer.
Furthermore, we wanted to create a haven where the patients' collected information was not jeopardizing and it was still private in whichever means it was communicated. We had to deal with this in two stages. The first stage was how we received the data collected. For our application, the data collection happened in the following way: we created a mobile application of which we utilized the microphone sensor to collect audio data which our system is built upon. Since our framework is built upon having audio data, we developed a system which as soon as the data is recorded on the phone it is able to be deleted and instantly thereafter a connection is made with the Fog computer.
Our development was to create a server client implementation. On the Intel Edison, we developed an encrypted server that always listened for a connection and received an audio file. Once the file was received, we were able to proceed with the file for further processing. The second stage where the communication layer was implemented when all of the extraction of audio files were processed. At this stage, once the extraction of the all of the important information is processed, a connection with our MYSQL database is made. Once the connection is successful, we are then able to upload the extracted data to the correct table in the database based on the on the ssid (Service set identifier) of the Edison, which is unique for every Intel Edison.

SOFTWARE/DATA INTERFACE
In the software layer of the research thesis, we utilized numerous software packages and framework to complete the task. In this section we will explain each of the software/data interface that was utilized.
On the Fog computer, we used the following software to complete the task of voice processing, extracted data storage, data transfer and upload, etc.: representation but since we are using an embedded system that function is not available. We were able to use some of the functions necessary from this library and others we had to copy and customize to fit our needs. The library is The following functions are a part of the scripts [26]: ▪ general utility scripts (generalUtility) ▪ digital signal processing (DSP) functionality (dspUtil and myWave) ▪ interacting and utilizing Praat (praatUtil and praatTextGrid) ▪ functionality for creating graphs (matplotlibUtil)  Praat is a free scientific open source software package tool which is used for analysis of speech in phonetics. It is a tool that is commonly used by SLPs and linguistics. It provides multiple capabilities ranging from spectrographic analysis, articulatory synthesis, neural networks, etc [27].
 ftplib is a python module which defines and implements file transfer protocol class in python. Just as a normal FTP client, ftplib allows us to do things such as list directories, download and upload files from and to the server, and do multiple other tasks. The reason why we decided to use this is that it gave us the ability to save any of the audio file that the Fog computer processes on [28].
 TinyDB is compact and lightweight database that can be used for query processing for information extraction [29]. TinyDB is packaged in many different ways. One of the ways is that that it is also a module for python.
TinyDB, much like MySQL, offers a simple interface to perform queries just as one would perform with SQL. The reason that we used TinyDB instead of using other applications like mariaDB is because it allows for power efficient innetwork processing algorithms [29].

APPLICATION LAYER
The application layer of our I.o.T. based system for the research thesis involved making an application so that all users of our study could collaborate and interact with one another. In this section we explain the application we have been developing and intend to continually develop.
We decided to make an Android application for our study. The purpose for using an Android application was to allow the use of software which made data collection seamless with little effort, and a unique profile and task for each user. In our case there were three different kind of users since each user had different tasks to complete throughout various steps of the application process. Figure 6 shows the hierarchy of the user privileges for the three kind of different users for the desired design scope of the research thesis. In the flowchart, we can clearly see the design scope of the privileges that took place for each of the user and also the account authentication structure.
We used a restful service to program the application so that we could gather information. REST (Representational State Transfer) is an architectural style of information transfer using HTTP methods such as GET, POST, PUT, and DELETE to allow HTTP commands to perform different actions depending on the backend server programming implementation [30]. This is very useful and beneficial since it is widely used by all commercial devices. We implemented a restful service since we wanted to create an architecture where registration was possible, authentication was possible, and data gathering for visual purposes was also possible. To use the RESTful service, all we had to do was use the HTTP library embedded in Android.

DESIGN AND IMPLEMENTATION / FINDINGS
In this section, we explain and present all of the design and implementation of the research thesis from the Fog Computer, the Database, to the mobile application. We first explain the overall structure of the study and the involvements of each front and back end user.

OVERALL ARCHITECTURE DESIGN
For the overall research, the beginning idea and approach began with Fog computing only. As time went along we started to think bigger: to the cloud since it offers the ability to save massive amounts of information. We then realized that we had to create a complete I.o.T based system where all the platforms were developed and functioning. We realized that there had to be multiple users, end-users and front endusers. In figure 7, we see the overall structure that we set our goals on to develop in an overview method. In figure 8, we are able to see a more detailed and descriptive view of the overall design.

HARDWARE (EDGE)DESIGN AND IMPLEMENTATION
We developed the hardware layer Intel Edison as our main Embedded network connected board. There were numerous libraries, modules, and software that were installed on the Edison so that we could implement and program the overall hardware design and interface of the Fog Computer.
For the first programming task, we had to make sure that we had to transfer the recorded audio file by the patient user. To do this, we needed to understand a couple of programming concepts and protocols. These protocols were to use socket streaming using TCP wrapped in SSL/TLS (Secure Socket Layer/ Transmission Layer Security) sockets for security.
Sockets provide the communication framework for devices using different protocols such TCP and UDP which could then be wrapped in secured sockets. TCP, Transmission Control Protocol, is a networking protocol which requires guaranteed and reliable delivery and in order files. TCP is a protocol that is connection oriented and bidirectional which means both devices can send and receive files sent to one another.
Each point of the connection involves an I.P. (Internet Protocol) address and a port number [31] so the connection can be made with a specific device. Furthermore, (for the reasons of security on the privacy sake of the users' information that is being collected), we wrapped the TCP sockets in SSL Sockets. SSL (Secure Sockets Layer) is network communication protocol that allows for encrypted authentication for network sockets from the server and the client side. This connection protocol allows for the data transfer that server and client to remain private and secure. To complete this task, we used the python module: SSL and socket. Additionally, to create the certifications for the server and the client, we used a command line program called OpenSSL (an open source project that provides a robust, commercial-grade, and full-featured toolkit for the Transport Layer Security (TLS) and Secure Sockets Layer (SSL) protocols [32]).
Socket implementation with SSL : key, cert = "/home/root/Coding/certificates/server/privKey.key", \ "/home/root/Coding/certificates/server/server.pem" # Key and cert created with OPENSSL context.load_cert_chain Once all the SSL certification keys (for client and server being the Android device and Intel Edison respectively) were built, we just ran the secure sockets on the server (Intel Edison) and continuously listened for a connection so that we can receive a file.
We then renamed the file with the date and time so that it can be ready for processing (See code above with comments).
As soon as the audio file was completely transferred, the connection is closed and the processing begins. For the processing and analysis, we utilized the Christian Herbst python library and praat tool which the python library takes advantage of (see chapter 3, section 3.2.2). The audio used for the development and implementation are described in table 3. Some of the processing parameters (averages of parameters) that we calculated from the audio files are:  Perceptual Loudness: ▪ Loudness is the perceived intensity of an audio signal. It is a mathematical quantity based on various auditory models.
 Zero-Crossing Rate: ▪ Zero crossing rate is used to analyze and recognize voiced-unvoiced parts of speech. It is a number of times the speech signal magnitude crosses the value of zero [33].

 Spectral Centroid:
▪ Spectral centroid computes the center of gravity, the frequency area that most of the signal energy is focused. It gives indication to distinct sounds that are produced which sound very 'bright' [34]  Short-time Energy ▪ Short-time energy was used to provide a representation that reflects amplitude variations of unvoiced and or voiced segments of an audio file [33]. Other parameters using the Christian Herbst python library that we were able to calculate were: Jitter ratio, average harmonicity, period variability index, root mean square, and fundamental frequency. Parameters such as jitter, shimmer, shimmer percentage, and many more will be additional parameters to be processed in the future.
We are thankful to the patients with Parkinson's disease for their co-operation in using the EchoWear and FIT technology. The presented work is supported by a grant The heart of the matter of the back-end design is the relational database development. We designed a database that revolves not around the users of the I.o.T.
based system but also one that the Fog computer could easily interface with the database. The design concept of the database for our system was at first based on the users. We created three tables that were only about the users (patients and healthcare providers). One table was created for the data extraction information of the patient. The following will explain each table and all of their field content and attributes.
Model of table with description for each filed in the Tables of the database:  Users Table: ▪

APPLICATION DESIGN/IMPLEMENTATION OF FRONT END
For the front-end user implementation, as previously mentioned, we designed an Android application so that both users and healthcare providers could interact with one another. Also, as mentioned before, we developed our app to take advantage of our web service by using the REST protocol (see chapter 3, Section 3.3 for more information).
The reason we used REST is because we don't have complicated data operations taking place and it is simpler to implement for future development. For every REST request of data information gathering, we returned a JSON ((JavaScript Object Notation) which is a format of data-interchange between programs [37]) object with information that we deciphered in the android application.
The application allowed for the access to the two kind of users for almost every action except with a couple of exceptions as seen in the figures below. Both of the patients and healthcare providers are allowed for login and viewing their profile, however the profiles are different in it that the doctor allows for a patient to be registered  The results obtained while conducting the research was produced as a result of the processing that we were capable to program on the Intel Edison. Some of the results from the processing audio files included calculating the Intensity of the audio files. In figure 11 we see the instantaneous loudness of speech signals computer by the Fog computer. The figure illustrates five different normal 'ah' pitch audio files (see table 4 in section 4.2 for explanation) throughout a week that the intensities were calculated from. Furthermore, since we had the ability to conduct multiple experiments from Parkinson's patients in different instances of time, we were able to also visualize the results of intensities. Figure 12, shows an example of multiple exercise audio files with low and high pitches for a 2-week time period (see table 4 in section 4.2 for explanation).  Other clinical speech features such as zero crossing rate (ZCR), the spectral centroid (SC), and short-time energy (STE) (for more information on the features, please review section 4.2) were also parameters that we were able to successfully process from the audio files. As a result, figure 13 illustrates an example of an audio file's amplitude signal and the above mentioned features calculated. Furthermore, just with the intensities calculated for numerous audio files, we did the same with ZRC, SE, STE parameters where the average for each audio file is calculated. Figure 14 illustrates the averages of the SRC, SE, and STE parameters that were computed on the Fog computer. The results obtained while conducting the research can be seen above. The results provide and enables us with the the analysis of speech signals in remote nonclinical settings such as homes and possibly other locations with network connectivity.
To compute the above results, we utilized the Intel Edison as the Fog computer with a service-oriented architecture to collect, process and extract clinically-relevant features, and store the information extracted. We demonstrated as part of the results from our research that the Fog computer in an I.oT. based system provides great computational intelligence, configurability, and interoperability. As a proof this, we benchmarked the calculations of the clinical features and the results are shown in Table 6.  Table 6. Benchmarking the fog computer for computing the clinical speech features: zero crossing rate, special centroid, and short-time energy

CONTRIBUTIONS
The goal of this research was to design a full course I.o.T platform from the ground up involving every layer implementation. Our aim was to develop an I.o.T service surrounding a database-centric approach with multiple user interactions. We decided to develop an application also revolving the database-centric approach.
Furthermore, most importantly, we developed the I.o.T system with Fog computer in mind to handle the data collection, processing, storage, etc. Essentially we utilized the Fog to be the main hub of our research. From first processing audio files which was capable of handling complex signal processing algorithms such as psychoacoustic analysis to extract clinical features, to storing the raw data and features that are queryable by the cloud and on the Fog by demand. The Fog is a useful layer in the platform that can have plenty of benefits especially in machine-to-machine interaction.

FUTURE RESEARCH
Our I.o.T. based system includes all of the functional components that are required, however, much work still needs to be done to perfect the system to function seamlessly. The following is a list of content and direction that the research can take: ➢ Although security is implemented at audio data transfer, upgrading the encryption would be beneficial ➢ A new form of Edison search and connection to send data needs to be implemented instead of giving the exact I.P. for the Edison.
➢ The android application could implement some of the encryption instead of server code implementing this since information could be hacked when an http request happens at login or register.
➢ Create an actual API for the mobile application to use instead of making a single request every time are posting to requesting data.
➢ Implement a notification system that the patient is notified when to complete a task.