Mohamed Fawzan Hussain, Year 3.
In 2016, it was estimated that over 85,556 people live with a Spinal Cord Injury (SCI). People with Spinal Cord Injuries (SCI) often struggle to move certain muscle groups and overcome barriers for comfortable living. Through using an Electrooculogram (EOG)- based Human Computer Interface (HCI), people with SCI can use their eye movements and blinking patterns to comfortably control things such as a robotic hand. This led the researcher to obtain EOG signals from volunteers and see if there are any correlations between the data using machine learning. The researcher found that the Machine Learning Algorithm consistently predicted the ‘Left Movement’, regardless of the input. This demonstrates that with more datapoints, these signals can be used as a novel method to potentially control a robotic hand.
According to Praxis Research Institute, it is estimated that there are 85,556 people living with a Spinal Cord Injury (SCI) in 2016 and an estimated annual increase of 4,529 cases of SCI in Canada. As a result of their disability, they often face several barriers for comfortable living, which may include limited interactions in social environments, respiratory functions, blood circulation and the ability to move certain muscle groups (Praxis Research Institute, 2019). In addition, individuals living with SCI in BC can face an estimated lifetime cost of $1.6 – $3 million (Praxis Research Institute, 2019).
While there is no imminent cure for Spinal Cord Injuries, there have been many technologies available for people living with SCI, including exoskeletons, stem cells and functional electrical stimulation; as a method to stimulate the physiological functions lost (Spinal Cord Inc.). These have been proven to be effective methodologies in assisting people with SCI to overcome physical barriers. However, many of these devices such as exoskeletons can “cost in the region of $80,000” and often involve invasive procedures to implant these technologies (SCI Progress).
One feasible solution to help people with SCI have an increased sense of mobility freedom is by using an Electrooculogram (EOG)- based Human Computer Interface (HCI). This is a system that uses electrooculogram (EOG) readings from eye movements to interface with a computer that performs external actions (i.e. selecting a key on a keyboard or lifting a robotic hand) and provides real-time feedback to the user with minimal delay. Similar applications of using an Electrooculogram (EOG)- based Human Computer Interface (HCI) includes research in eye movements and development of a robotic eye with natural human eye movements (McGill, 2020).
Research from the University of British Columbia showcase that it is possible to classify eye movement signals based on distinctive patterns (Gunawardane, P.D. S.H., et al, 2019). This demonstrates that these distinctive signals could be used as a trigger to activate external actions.
The objective of this project is to accurately classify EOG signals (namely eye movement and blinking signals) from volunteer participants and understand any correlations between these signals through the use of a Machine Learning Algorithm.
The advantage to an EOG based system is that SCI patients often retain the ability to control their eye movement in the same manner as people without SCI; as well as it being a low-cost and non-invasive solution (Zhang et al. , 2019).
Materials and Methods
Before participating in the experiment, volunteer participants were asked to thoroughly review the Letter of Information and provide a signature on the Letter of Consent, where applicable. Once signed, each volunteer participant was given a Participant Identification Number as to anonymize the identification of the volunteer participants in project documentation and presentation.
Then, the researcher gave a detailed description of the project, described the impact of this low-cost, non-invasive system, explained the purpose of the assessment: to understand the correlations between data acquired from the EOG and an individual’s eye movement and blinking patterns and answered any questions, comments or concerns.
Following this, participants were asked to apply a rubbing alcohol pad and EOG Electrode Gel to enhance the quality of readings of the electrode and to detect electrical activity from underneath the skin. Once, the rubbing alcohol and EOG Electrode Gel (signa gel) were applied, participants were fitted with the OpenBCI EOG alongside two reference electrodes.
Figure 1: OpenBCI EOG
Now the participant was ready to complete the assessment. EOG readings were recorded on volunteer participants as they blinked twice, moved their eyes to the right and back and moved their eyes to the left and back. Each movement was conducted thirty times by the participant, with a short resting interval in between each completion of a movement done thrice.
The EOG Readings were recorded using the OpenBCI GUI Application and saved to a text file, encapsulating all readings for further analysis.
This data was manually classified as either left eye movement, right eye movement or double blinked, through a custom Python program the researcher created. The researcher understood what these movements looked like through observations made and recorded patterns. In addition, these patterns were observed in other participants as well. All recorded data were passed through a bandpass filter (a lowpass cutoff of 5Hz and a highpass cutoff of 15 Hz, in this project), responsible for removing external electrical noise and focusing solely on the brain’s neural activity.
Once manual classification was completed, the researcher saved the data to a PKL file (a Python file type that allows for serialization of files and deserialization when being read). (FileInfo, 2017)
Figure 2: Sample Graph of a Double Blink from participant
Figure 3: Sample Graph of Eye Left Movement from participant
Figure 4: Sample Graph of Eye Right Movement from a participant
Next, a multi-class classification logistical regression machine learning model, developed in Python was built to accurately classify the EOG data based on the given categories. The researcher used 75% of the dataset to train the model and 25% of the dataset to test the model with a random state of 14. Training and testing could be thought of as studying for a test and doing a test, respectively whilst being prepared for any random question to appear.
To analyze the performance of the machine learning model, the researcher used the mean absolute error.
Figure 5: Mean Absolute Error Equation
The mean absolute error equation is most often described as a methodology in statistics which “measur(es) the difference between two continuous variables” (E | M, 2018). In this project these two continuous variables would be the raw EOG Data versus its classification type. Through using this equation with the defined parameters mentioned above, the researcher attained a Mean Absolute Error Score of: 0.6214096898918962.
In addition, a Confusion Matrix was developed to understand the consistency in decisions that the Logistical Regression Machine Learning Algorithm took.
Figure 6: Confusion Matrix with Normalization
Through analyzing the normalized confusion matrix, it shows all the datapoints given, the algorithm consistently classified all data as the ‘Left’ movement. This demonstrates that the machine learning algorithm is not able to accurately classify the EOG data used in this project.
Some significant errors in the machine learning process included incorrectly labelled data, unbalanced datasets [i.e. more data was classified as one movement compared to the other labels] and incorrect use of machine learning algorithm processes. An abundance of these errors causes the machine learning algorithm to “learn” these falsities as truths and create an undesired output, as witnessed in this project.
Before obtaining EOG-related biodata from volunteer participants, the researcher gathered these recordings on himself. Through this process he visually observed certain patterns that were associated with each movement and correlated this visual data to a certain movement (i.e. moving eyes to the left, blinking twice or moving eyes to the right). Through analyzing participant’s data, he observed similar patterns through each of the classifcations, especially between blinking twice (Figure 1) and moving eyes to the right (Figure 3).
This led the researcher to do a validation test with volunteer participants to see if this assumption is accurate. Through classifying the randomly selected graphs, the participants combined, correctly identified 17 out of 30 images. These participants misidentified Figure 1 for Figure 3, 5 out of 17 times and Figure 3 for Figure 1, 6 out of 17 times. These results demonstrate that the machine learning model may have had difficulty identifying distinct features of Figure 1 and Figure 3; thus, leading to increased confidence in its classification of Figure 2.
In addition, the researcher observed blinking patterns while participants moved their eyes to the left or right. This presents extreme variation in the data and may have presented a challenge in the process of machine learning classification.
Nonetheless, the researcher firmly believes that an EOG-based Human Computer Interface is not only limited to people with SCI but can be expanded to people with disabilities for use in a multitude of possible scenarios (i.e. analyzing signals to control a robotic prosthetic, select a key from a keyboard, control a light switch etc.). The major advantages of such technology are the inexpensiveness, non-invasiveness, and ease of setting up whilst the main disadvantages of this technology include limited research and analysis in using EOGs as a methodology to overcome barriers for people with disabilities (FutureToday, n.d.).
An alternate but similar technology to the EOG-based Human Computer Interface is a Brain Computer Interface. Brain Computer Interfaces present extreme non-invasiveness and thorough documentation. However, the major disadvantages to this system are expensive initial prices ranging from ‘$5,000 to 10,000’ (Shih et al, 2012), minimal security in securing biodata and that blinks heavily influence the readings inputted from a Brain Computer Interface. (Manders et al., 2019)
While EOG recognition offers fewer potential choices than BCI, this system is a lot more realistically priced. This would remove financial barriers that many people with SCI face as individuals living with SCI in BC can face an estimated lifetime cost of $1.6 – $3 million (Praxis Research Institute, 2019).
Even though this project is a novel concept, the researcher believes that he can work on improving parameters for the machine learning algorithm to “learn” better and correlate these signals to move a robotic hand in ‘real-time’, helping people with Spinal Cord Injuries overcome physical and financial barriers they face during their day-to-day lives.
“Band-Pass Filters: Filters: Electronics Textbook.” All About Circuits, www.allaboutcircuits.com/textbook/alternating-current/chpt-8/band-pass-filters/.
Bulling, Andreas. “Eye Movement Analysis for Activity Recognition Using Electrooculography.” Eye Movement Analysis for Activity Recognition Using Electrooculography – IEEE Journals & Magazine, 8 Apr. 2010, ieeexplore.ieee.org/document/5444879.
E | M. “Mean Absolute Error ~ MAE [Machine Learning(ML)].” Medium, Medium, 22 Feb. 2018, medium.com/@ewuramaminka/mean-absolute-error-mae-machine-learning-ml-b9b4afc63077.
FileInfo. “.PKL File Extension.” PKL File Extension – What Is a .Pkl File and How Do I Open It?, 9 Mar. 2017, fileinfo.com/extension/pkl.
Future Today. “Advantages and Disadvantages of Electro-Oculography (EOG).” Advantages and Disadvantages of Electro-Oculography (EOG), www.realexposer.com/2010/01/advantages-and-disadvantages-of-electro.html.
Gunawardane, P.D. S.H., et al. “An Oculomotor Sensing Technique for Saccade Isolation of Eye Movements Using OpenBCI.” ResearchGate, Oct. 2019, www.researchgate.net/publication/336903635_An_Oculomotor_Sensing_Technique_for_Saccade_Isolation_of_Eye_Movements_using_OpenBCI?mc_cid=ab266f9cfa&mc_eid=ddd6f840e7.
Horst, Jeremy. “Brain-Computer Interfaces: Advantages and Disadvantages.” Brain-Computer Interfaces: Advantages and Disadvantages, 1 Jan. 1970, jhorst1.blogspot.com/2009/10/brain-computer-interfaces-advantages.html.
Manders, Neil, et al. “Exploring the Benefits and Risks of Brain Computer Interface.” Connects Decision Makers to You, 2019, www.bbntimes.com/technology/exploring-the-benefits-and-risks-of-brain-computer-interface.
McGill University. “Biological Signals Acquisition.” The McGill Physiology Virtual Lab, 2020, www.medicine.mcgill.ca/physio/vlab/Other_exps/EOG/applications_n.htm.
Narkhede, Sarang. “Understanding Confusion Matrix.” Medium, Towards Data Science, 29 Aug. 2019, towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62.
Praxis Spinal Cord Institute. “Understanding SCI.” Praxis Spinal Cord Institute, praxisinstitute.org/living-with-sci/understanding-sci/.
SCI Progress. “Spinal Cord Injury Technology Advancements.” SCI Progress, 30 July 2019, sciprogress.com/spinal-cord-injury-technology/.
Spinal Cord Inc. “SCI Technology Advancements.” Spinalcord.com, 2020, www.spinalcord.com/sci-technology-advancements.
Shih, Jerry J, et al. “Brain-Computer Interfaces in Medicine.” Mayo Clinic Proceedings, Mayo Foundation, Mar. 2012, www.ncbi.nlm.nih.gov/pmc/articles/PMC3497935/.
Zhang, Rui, et al. “An EOG-Based Human–Machine Interface to Control a Smart Home Environment for Patients With Severe Spinal Cord Injuries.” IEEE Journals & Magazines, 2019, ieeexplore.ieee.org/document/8356680.
Figure 7: Letter of Information Page 1
Figure 8: Letter of Information Page 2
Figure 9: Letter of Informed Consent