Development of Low-Cost system for Myoelectric Prosthesis Control

The upper limb prosthetics field has advanced to where artificial hands can mimic human motions based on electrical signals emitted from muscular activation and can even open and close proportionate to the amplitude of signal they receive. However, these prostheses are very expensive, with some costing up to $75,000 USD. 3D-printing has greatly lowered the cost of manufacturing multiarticulate prosthetic hands, but there is currently no control system that can provide these inexpensive hands with proportionate, myoelectric control of more than one degree of freedom (DOF) at a similar price reduction. This project aimed to develop a low-cost control system that can interpret myoelectric data for multiple DOFs and control a prosthetic hand. The system was developed using a combination of inexpensive microcontrollers with advanced control algorithms. Once it was prototyped, we tested the system's signal acquisition capabilities against a research-grade system by comparing both systems' signal-to-noise ratios (SNR). We also investigated the capabilities of the low-cost system's proportionate control of multiple DOFs and compared this against the research-grade system, using root mean squared error (RMSE) as a metric. We found that the SNR of the low-cost system was statistically no worse than 44% of the SNR of a research-grade system. We also found that the RMSEs were typically a little better, and no more than 6% worse than the research-grade system's RMSEs. However, RMSEs were still high (therefore worse) due to a restriction of electrode channels used (six vs. typically greater than 32). Future work will focus on increasing the channels and therefore improving the control. With a materials cost of ~$675 USD, this prototype takes an important step towards providing high-end prosthetic technology at a relatively affordable cost.

DEVELOPMENT OF A LOW-COST SYSTEM FOR MYOELECTRIC PROSTHESIS CONTROL by Sridharan Radhakrishnan A Senior Honors Thesis Submitted to the Faculty of The University of Utah In Partial Fulfillment of the Requirements for the Honors Degree in Bachelor of Science In Biomedical Engineering Approved: _______________________________ Gregory A. Clark, PhD Thesis Faculty Supervisor _____________________________ David W. Grainger, PhD Chair, Department of Biomedical Engineering _______________________________ Kelly W. Broadhead, PhD Honors Faculty Advisor _____________________________ Sylvia D. Torti, PhD Dean, Honors College May 2020 Copyright © 2020 All Rights Reserved ABSTRACT The upper limb prosthetics field has advanced to where artificial hands can mimic human motions based on electrical signals emitted from muscular activation and can even open and close proportionate to the amplitude of signal they receive. However, these prostheses are very expensive, with some costing up to $75,000 USD. 3D-printing has greatly lowered the cost of manufacturing multiarticulate prosthetic hands, but there is currently no control system that can provide these inexpensive hands with proportionate, myoelectric control of more than one degree of freedom (DOF) at a similar price reduction. This project aimed to develop a low-cost control system that can interpret myoelectric data for multiple DOFs and control a prosthetic hand. The system was developed using a combination of inexpensive microcontrollers with advanced control algorithms. Once it was prototyped, we tested the system’s signal acquisition capabilities against a research-grade system by comparing both systems’ signal-to-noise ratios (SNR). We also investigated the capabilities of the low-cost system’s proportionate control of multiple DOFs and compared this against the research-grade system, using root mean squared error (RMSE) as a metric. We found that the SNR of the low-cost system was statistically no worse than 44% of the SNR of a research-grade system. We also found that the RMSEs were typically a little better, and no more than 6% worse than the research-grade system’s RMSEs. However, RMSEs were still high (therefore worse) due to a restriction of electrode channels used (six vs. typically greater than 32). Future work will focus on increasing the channels and therefore improving the ii control. With a materials cost of ~$675 USD, this prototype takes an important step towards providing high-end prosthetic technology at a relatively affordable cost. iii TABLE OF CONTENTS ABSTRACT ii INTRODUCTION 1 BACKGROUND 4 METHODS 10 RESULTS 16 DISCUSSION 23 ACKNOWLEDGEMENTS 29 REFERENCES 30 iv INTRODUCTION Each year, 6,000 Americans undergo upper-limb amputation [1]. If they wish to work, grow, or adjust to society with bimanual tasks, they often must obtain a prosthetic hand. However, the high cost of viable upper-limb prostheses places a considerable financial strain on this population and process [2]. One advanced prosthetic option is a proportional multiarticulate hand, which allows the user to change grips and finger positions. Pröbsting et al showed that multiarticulate hands improved amputee’s usage of their prostheses [3]. A key feature of some multiarticulate hands is myoelectric control. These prostheses can detect electromyographic (EMG) activation levels (electrical signals from muscle activation) and change their grip based on that pattern [4]. Further, some can detect changes in EMG activation and change their grip strength proportionally to match [5]. Proportional control is favorable as it allows a user to complete fine motor actions such as pick up fragile objects, use a fork and knife, and other essential acts of daily living intuitively [6]–[8]. Unfortunately, these devices often come with high price points, with some costing up to $75,000 USD [2]–[4]. 3D printing has allowed companies such as OpenBionics, Mand.ro, and BLINC Lab to develop less expensive multiarticulate hands, some of which are open source [4], [5], [9]. Nonetheless, the issue is that multiarticulate, myoelectric, and proportional hands typically only control one degree of freedom (DOF), and remain very expensive [5]. Sreenivasan et al. developed an ultra-low-cost option, but it is a claw design that only opens and closes (i.e., one DOF) [10]. 1 A different approach allocates myoelectric, proportional control of a prosthetic hand to a separate control system, which then just instructs a multiarticulate hand. This allows for higher processing power, which extends proportional control past one degree of freedom [11]–[15]. This combines the advantages of having multiple grips [3] and proportional control [6]. Examples of these systems include Ripple Neuro’s Grapevine Processors (Grapevine Neural Interface Processor; Ripple Neuro LLC, Salt Lake City, UT USA). However, this added functionality comes with an added cost. In addition to purchasing a prosthetic hand, systems like Ripple Neuro’s Processors can cost around $64,000 USD [13]. Since payor entities such as Medicare cover only 20% of approved prostheses [16], this level of advanced proportional prosthetic technology remains inaccessible to many amputees. Even with access, the financial risk of damaging and maintaining such expensive equipment causes buyers to abandon their use [17]. As mentioned earlier, 3D-printing can lessen the cost of prosthetic hands, but there is no analogously inexpensive control system to provide myoelectric, proportional control of multiple DOFs. Therefore, we aimed to design a less expensive myoelectric proportional control system to fill the niche between high-end myoelectric prosthetic control and financial accessibility. This low-cost system (LCS) was intended to provide intuitive, proportional control of at least three degrees of freedom. In addition, it needs to control hands in real-time. Finally, the LCS should be portable. We used economical microcontrollers [18], [19] and established prosthetic control algorithms [15], [20], [21] to provide proportional control. 2 Figure 1. This diagram shows how this system turns muscle-activation into prosthetic movement. Generally, electrodes collect surface EMG data from a forearm. The control system then processes that data and translates it into instructions for a prosthetic hand. Successful development of an economic system with proportional control will improve the accessibility of advanced prosthetic technology to patients that need it. Better financial access would ensure that this technology is available to improve amputee quality of life. 3 BACKGROUND Prostheses have existed for millennia. Their history is rich, ranging from aiding Egyptian priests and leaders, Roman generals, medieval knights, revolutionaries, pirates, and many others [22], [23]. They have evolved from crude replacements fashioned out of wood and leather to the elegant machines seen today. However, some modern solutions have not come far from their origins, just improved their efficiency. For example, the firstrecorded example of a prosthetic arm capable of movement was in the 1500’s. “Le Petit Lorrain” was a prosthetic hand fashioned out of iron, attached to a series of catches, springs, and a harness that allowed a French army captain to manipulate the prosthetic and enter battle [22]. This is not unsimilar to modern-day body-powered prostheses. These devices rely on the motions of intact muscles and joints in the user to manipulate a handpiece via a harness. The modern body-powered prosthesis can be traced back to 1818 [23]. In addition, cosmetic prostheses have become steadily more realistic since their creation, and as such are still a viable option for amputees today. The relatively new technology in myoelectric prostheses arose in the 1940’s, with the advent of electromyographic signals being used to control an externally powered prosthetic hand [23], [24]. This option gained clinical significance rapidly, and today is a common option for amputees. The options for upper-limb amputees today range in functionality and price. Cosmetic prostheses are purely to pass as non-amputated and serve very limited functional use. However, these are typically the least expensive option for amputees at around $3,000$5,000 USD [1], [2]. Body-powered hooks allow for limited control of one degree of freedom but cost around $10,000 USD [2]. Myoelectric options provide the most user4 friendly control, but take time to train users on proper use, require an external battery, and are typically far more costly than other options (>$30,000 USD) [1], [25]. The cost barrier is only increased as myoelectric or neural prostheses increase the level of control afforded to the user. Despite the higher price, myoelectric prostheses have the greatest potential in giving amputees intuitive, user-dictated control of their devices. Greater affinity with their prostheses means bridging the gap between viewing it as a machine and viewing it as part of themselves [14]. In general, amputees are able to complete more tasks with myoelectric control than with body-powered hooks [3]. Whereas a body-powered system requires extra motion, a myoelectric prosthesis only requires residual muscle activation, which is more similar to intact hand function. Currently sold myoelectric options typically control one degree of freedom at a time [5], with the ability to control multiple DOFs only being available at exponentially increasing cost [13], [26]. These market options also have the functionality of switching what degree of motion is in control. For example, a user could use myoelectric control to open and close a fist, then press a button and change the grip to flex and extend the index finger. While multiple grips are desirable [3], [27], [28], having to press a button to change what grip the prosthetic makes is less intuitive than the prosthetic recognizing what grip is being attempted. That is the advantage of multiple DOFs control systems such as the DEKA Arm paired with the Ripple Nomad processor. When the processor is encoded with an advanced classification algorithm, it is able to determine user intent and translate that into prosthetic instructions for the prosthetic arm [11], [20], [21]. Again, while these 5 systems hold the greatest potential for intuitive prosthetic control, they are also considerably expensive, costing upwards of $100,000 USD [1], [13], [26]. The power and cost of this level of technology shows the branching point in current approaches in the muscle-activated prosthetics field: economization and refined control. The push for economization comes from the large patient base that cannot access prosthetic technology due to the cost barriers previously mentioned. These barriers lead to a few unique issues. The most apparent issue is that amputees across the world are left without advanced control and in many cases, no prosthesis at all. In developing countries, innovation must focus on ultra-economization and durability. Sreenivasan et al. attempted to focus on these requirements with their prosthesis [10]. Their device utilizes stretch resistors instead of electromyography. Since muscle flexion shortens and thickens the muscle, that signal can be used to proportionately control a claw. The more the residual muscles shorten, the more the claw opens or closes. It essentially functions as a one DOF proportionate prosthesis and utilizes 3D-printing for materials and an Arduino Nano as the controller. Their work shows that economization is possible, but it is limited to one DOF. Another issue discusses relative function of myoelectric prostheses to bodypowered prostheses. As mentioned earlier, body-powered prostheses typically cost approximately $10,000 USD where myoelectric devices cost approximately $30,000 USD. In addition, body-powered prostheses are more durable and according to some amputees, easier to control. The increase in cost is not worth the tradeoff in function for many amputees [29]. Another unique issue is that even people that can afford and buy these devices commonly underuse them, an occurrence dubbed “device abandonment.” Abandonment 6 arises from several factors including low degrees of controllable freedom and extremely high cost [17], [28]. Essentially, due to the high cost-to-functionality ratio, many amputees opt not to use their prosthesis. To them, the risk of damaging their device and undergoing costly repairs outweighs the benefit the device provides [17]. Economization would allow for use across the world as a better option than bodypowered prostheses and would lower the risk associated with owing a myoelectric prosthesis. As a result, amputees would feel comfortable utilizing their hands more often. The recent innovation of 3D-printing has drastically reduced manufacturing costs for several previously expensive devices. 3D-printed hands can now be printed and assembled by anyone with access to a 3D-printer [9], [30], and companies such as Open Bionics have reduced the cost of myoelectric devices through 3D-printing. Their Hero Arm [5] costs around $3,000 USD [31] and allows for myoelectric, proportional control of one DOF. Users can change the active DOF with a button-press, for example switching from making a fist to flexing and extending an index finger. The Hero Arm is self-contained, meaning a user only needs to slip the device on with no need for outside cabling or accessories. The 3D-printing also allows for customization in shape at a very low price and has also been used to improve the aesthetics of the device. It is a popular option amongst children, who rapidly grow out of prosthetic devices, therefore needing low-cost options. This is one of the most competitive devices on the market, but still can only control one DOF at a time. However, the Hero Arm showcases the use of 3D-printing to lower the cost of prosthetic systems. It simply needs more refined and capable control. Several groups are working to improve prosthetic control by increasing the number of controllable DOFs. As demonstrated above, many myoelectric prostheses use direct 7 proportional control to EMG data. In current market systems, that data comes from only one or two muscle groups. In the case of a transradial amputee, they might be trained to use their wrist flexors and extensors to control the prosthetic grip. In order to change grips, they would press a button before continuing. Increasing the number of controllable DOFs at a time means developing algorithms to take in more EMG data and computationally predicting what motion is being made. Developers use multiple channels of EMG data targeting several different arm muscles in order to increase the accuracy of prediction. The data are fed to a control algorithm, which outputs predicted motions and instructs the artificial hand to move. Different research groups have implemented several classification algorithms, such as the Kalman Filter [11], [20], [21] and the convolutional neural network [15], [32]. These control algorithms make it possible to classify several different motions based off of the incoming EMG data. A user would be able to control one DOF, then switch to another without pressing a button or thinking about it at all. That is the true importance of high-quality prosthesis control. The less a user thinks about their prosthesis, the more natural it feels. Current options for refined control are limited to clinical trials at the moment but include the DEKA Luke Arm paired with a Ripple Nomad processor. The DEKA and Ripple combination is also advanced enough to provide a sense of touch to the user, something other systems have yet to achieve. Most other applications of high-end prosthetic control are limited by needing a non-portable computer to run software [15], [30], [32]. While these systems provide precise and accurate control of multiple DOFs, they are either too expensive, impractical, or both. It has been demonstrated above how many devices exist at both ends of the costto-performance spectra of myoelectric prosthetic systems. The benefits of both ends are 8 clear: better control equates to better performance in day-to-day activities, and lower cost increases the accessibility of the technology. Amputees deserve the best of both worlds, yet there is a lack of devices that achieve both goals and provide high-quality control at a low cost. The control system developed through this research aims to fulfill these criteria. In order to compare to the devices mentioned above, this system would need to control multiple DOFs simultaneously, be portable, be compatible with multiple different prosthetic hands, and most importantly cost less than competing devices, even with the cost of a prosthetic hand included. 9 METHODS This project’s criteria were set up based on the identified need of an advanced and economic prosthetic control system. Tests were conducted at key steps of the prototype system’s process to verify functionality. Verification was completed by comparing LCS performance to a high-cost system (HCS) (Grapevine Neural Interface Processor; Ripple Neuro LLC, Salt Lake City, UT USA). The methods below detail the specific parts, theories, and tests utilized to produce the prototype system. Design Criteria Development Criteria for the system’s function were based off of research-grade control systems while emphasizing low-priced elements. The system should 1) cost less than the Ripple Nomad (~$64,000 USD) [3], 2) rely on a control algorithm geared towards proportional control, 3) process data and control the hand in real-time (~40 ms update rate) [14], 4) control greater than one DOF, and 5) be portable. Additional criteria arose during development as a result of decisions made to address the 5 primary needs. These are 2a) a capacity to store several 15+ minutes of surface EMG (sEMG) recordings, and 5a) a long battery life, which we determined as greater than eight hours. Methods of Signal Acquisition and Analysis Data were recorded using sticky snap electrodes, the Backyard Brains sEMG acquisition device and an Arduino Mega. The electrodes were placed on the proximal forearm of a human participant. Six electrode cables were attached to these eight electrodes and were plugged into the sEMG acquisition device (Muscle Spikershield Pro; Backyard Brains, Ann Arbor, MI, USA). They were single ended: the reference and ground for all 10 six cables were tied together into a single reference and a single ground electrode. Positioning of the electrodes vary depending on the person’s anatomy. The general rule was three electrodes target the extensor muscles, three for flexors, and the reference and ground electrodes placed on the distal, posterior area of the upper arm [9]. A signal-to-noise ratio (SNR) comparison between the LCS shield and a researchgrade system [13] was conducted. The goal was to determine whether the raw sEMG signal retrieved by both systems was comparable. The experiment was setup in the following manner: A human non-amputee participant would have the eight sticky electrodes placed on their forearm. The Spikershield comes with six aux-to-alligator clip cables. These alligator clips were attached to the electrodes, and all of their reference and ground clips were attached to the single reference and ground electrodes. Simultaneously, the researchgrade system’s cables were attached to the electrodes. Both systems send the recorded signals to a computer, which saves the data as .csv files. Six data channels were recorded on both the research-grade system and the Spikershield system, along with timestamps for each system. The participant is prompted to perform three different movements: 1) rolling their wrist, 2) making a fist, and 3) wiggling all of their fingers, dubbed “piano fingers.” Each of these motions were performed for seven seconds, followed by a ~10 second rest period. This test was conducted with three intact (non-amputee) participants, three times each. The means and standard deviations were collected. The csv data from both system recordings were synchronized in MATLAB using the time recordings. The data were partitioned into four sections; one for each movement and the rest period. SNR was calculated using Eq. 1 (power of the active sEMG data divided by power of the rest period): 11 SNR = (Poweractive/Powerrest). Eq. 1 A two one-sided t-test for equivalence (TOST; [33]) was conducted between the NIP and LCS for each movement (wrist, fist, and piano fingers). The TOST was used to determine the minimum equivalence interval where the recordings from the LCS and HCS were statistically equivalent (α = 0.05 and N = 18 electrodes). Signal Processing Pathway Once the data are acquired, the Arduino Mega sends it to the Raspberry Pi. The communication utilizes a serial communication port running at a Baud Rate of 2 megabits per second. The transmission of each line of data takes ~2 ms. The Pi fills a 300 ms buffer with the sEMG data, then takes the absolute value and average of the data at ~25 Hz, close to a common real-time processing rate [14] effectively smoothing it, as represented in Figure 2. In addition to smoothing the data, the Pi also takes the six original channels and creates 6 choose 2 additional channels of data. These additional channels of data are created by taking the differentials of any two original channels. By doing this, the system has 6 + 15 = 21 total channels of different sEMG information to use in its decoding. After this smoothing and extrapolation, the information is now ready for the decoding algorithm. 12 Figure 2. This graphic shows a representative raw EMG signal (top) and the smoothed EMG signal, aka feature (bottom)[14]. Methods and Analysis of Independent and Proportional Prosthetic Control The Kalman Filter control algorithm was chosen to satisfy the requirement for proportional control of multiple DOFs. “A Kalman filter is an optimal estimation algorithm used to estimate states of a system from indirect and uncertain measurements.” [11] A variant of this control algorithm common in proportional prosthetic control was implemented on the Pi [12]. The algorithm is used at the University of Utah Center for Neural Interfaces [11], [21] and was ported from MATLAB to Python. The Kalman Filter has two phases in its function: Phase one is training. During training, the Pi sends preprogrammed instructions to the prosthetic hand—for example, closing and opening a fist. A participant will then use their forearm muscles (residual muscle in amputees) to imitate the hand’s motions. The instructions sent to the hand as well as the sEMG data are collected simultaneously to .csv files. The filter then correlates the preprogrammed movements with the sEMG data it receives. It is then deemed ready for phase two: testing. During testing, the participant gains control. When the participant uses their muscles in similar patterns to the training session, the filter recognizes the sEMG patterns and sends 13 proportional instructions to the prosthesis. Training sessions can be programmed for whatever hand motions required (up to 6 DOF, depending on the prosthesis used). Training usually requires repeated motions, so the decode can have more information to determine which sEMG patterns match with which motion. Root Mean Squared Error (RMSE) was used to determine how well the decoding algorithm functions with a range of trained DOF. The setup was as follows: electrodes were placed on a participant forearm, and the participant was connected to the LCS. The participant then follows a training protocol. The first was two sets of five full thumb flexions and five full thumb extensions. The next was two sets of five thumb flexions and extensions, then five thumb oppositions and repositions. The third was thumb flexion, extension, opposition, reposition, then index flexion/extension. This continued for all six motions (thumb flexion, thumb opposition, index flexion, middle, ring, and little). Each of these trainings was conducted one time. As mentioned earlier, in each training, two sets of motions were completed. Then the process was repeated on the HCS. CSV recordings were produced from both the LCS and HCS sessions and were offloaded to a computer loaded with the same Kalman Filter algorithm. The algorithm used the first set of data (five flexions/extensions, etc.) to train. Then it would analyze the second set’s sEMG data and run a nonchronological testing phase, as all the data had been collected. RMSE would be calculated between the intended motion on each DOF, and the Kalman Filter’s estimation to compare how well they matched. In addition, RMSE was also calculated for unintended motion on each DOF. Unintended motion is defined as when the Kalman Filter estimates a motion on a DOF that the participant is not attempting to move. A TOST was used to 14 find the minimum interval for which the LCS and HCS RMSEs were statistically equivalent (with α = 0.05 and N = 6 participants). System Packaging and Portability To satisfy the requirement for portability, the system was assembled and packaged into a custom 3D-printed case. In addition, a power bank with 4.8 Amp output was purchased to power the Raspberry Pi and Arduino. The bank has the capacity to run the system for ~8 hours if fully charged. This battery is also housed in the case. An additional functionality that would be useful to the user is button operated training and testing. Having buttons soldered to the Raspberry Pi would allow the user to run the training and testing algorithms without manipulating the microcontroller. A custom cable was soldered, with SAMTEC ends for easy connection of electrodes. A prototype sleeve was developed using Anna Harding’s research [34]. The entire system is broken down in Figure 4 and shown on a person in Figure 5. 15 RESULTS Signal-to-Noise Ratio was Not Significantly Different between Low-Cost and High-Cost Systems For the setup of this experiment, the SNRs between the LCS and HCS were comparable. The SNRs (mean ± standard deviation) for digit movements for the LCS vs. HCS were 2.08 ± 0.83 vs 2.38 ± 1.67, respectively. Grasping SNRs were 3.69 ± 0.99 vs 4.51 ± 2.78, and wrist motion SNRs were 2.90 ± 0.72 vs 3.34 ± 1.90 (Fig. 3A). LCS SNR was statistically equivalent to the HCS within +0.45 or -1.05 SNR for individual digit movements, +0.36, -1.99 SNR for grasping, and +0.38, -1.25 SNR for wrist movements (p’s < 0.05, TOST). Therefore, the SNR of the LCS for individual digit movements and grasping was at most 44% worse than the HCS SNR, and at most 37% worse for wrist motions (Fig. 3A) [35]. Implementation of a Modified Kalman Filter can Provide Independent and Proportional Control Intended movement is defined as the DOFs move when the participant attempts to move them. Unintended movement is defined as movement in other DOFs than the DOF the participant is actively attempting to control. The LCS’s intended movement RMSE was at most 6% worse than the HCS intended movement RMSE for one trained DOF. The LCS RMSE was at least 4% better than the HCS RMSE for six DOFs (p’s < 0.05, TOST; Fig. 3B). Unintended RMSE of the LCS were at most 0.03 more than the respective HCS RMSE for all trained (one to six) DOFs (p’s < 0.05, TOST; Fig. 3C) [35]. 16 Figure 3. Comparisons between the LCS and HCS yielded similar results using six electrodes. Bars represent mean ± standard deviation. Values above the bars represent to the upper and lower statistical equivalence bounds relative to the HCS. The bounds were determined using two one-sided t-tests for equivalence (TOST, p’s < 0.05). A) The results from SNR analyses for three different motions: individual digital motion, a grasping motion, and wrist motions. B) The intended RMSEs were calculated for one to six DOFs. C) Unintended RMSEs are shown for two to six DOFs. The scale between intended and unintended RMSEs is roughly 10:1, respectively [24]. Design Criteria Were Achieved The total cost to build this system totaled ~$675 USD, 0.01% of currently available systems [13]. Table 1 summarizes how well the design goals were met. 17 Table 1. Design criteria and methods of achievement. GOAL ACTUAL Reduce Cost ~$675 USD Communicate between Arduino, Pi, Update Speed ~40 ms and Hand Store/Process Large Sets of sEMG Data Implement Intuitive Control Algorithm System is Portable Works with Multiple Hands System stores several 15+ minute sets of training data, storage expansion possible Modified Kalman Filter provides proportional control for 3+ degrees of freedom System in 3D-printed case, can be worn on hip or in a backpack. Portable battery provides 8+ hours of function. System works with HANDI Hand and Ada Hand, with potential to work with others. Materials Overview Table 2 outlines all of the materials required to build the resulting prototype as well as their costs. Utilizing inexpensive microcontrollers drastically lowers cost, but the price is bottlenecked by the EMG acquisition device, the Muscle Spikershield Pro. Table 2. This table breaks down the material costs of the system [24]. ITEM PURHCASED PURPOSE PRICE (USD) Muscle Spikershield Pro EMG acquisition and filtering/amplification Signal packaging and communication Control algorithms and hand control Data storage Portable battery $399.99 Other materials (cables, buttons, electrodes, etc.) Interfacing ~$60.00 3D-printing Portable case ~65.00 Arduino Mega 2560 Raspberry Pi 3b+ 32 GB Sandisk SD card Power Core 20000 Redux portable power bank 18 $35.50 $45.99 $19.99 $49.99 Final Design Functionality and Walkthrough The final prototype of the low-cost system (LCS) consists of the surface electromyograph (sEMG) acquisition hardware, microcontrollers to interpret the data and translate it to prosthetic motion, an eight-hour rechargeable battery to power the microcontrollers, and a 3D-printed casing to make the system portable. Figure 4 shows all the components as well as their layout within the casing. Items B-G are housed within the case. For the system to work, EMG electrodes must be attached to the user’s forearm in a manner where half of the six electrodes target flexor muscles, and the other half targets extensors (Fig. 4A). Once attached, the electrodes transmit sEMG signals to the EMG acquisition device, the Muscle Spikershield Pro (Fig. 4B). This device amplifies and bandpass filters the raw sEMG data with cutoff frequencies of 55 and 2500 Hz. The Spikershield is mounted on top of an Arduino Mega microcontroller (Fig. 4C), which receives the filtered data from the Spikershield. The Mega then relays the data to a Raspberry Pi microcontroller (Fig. 4D) at a rate of 1 KHz. The Raspberry Pi places that data into a buffer containing 300 ms of sEMG recordings. The absolute value of these recordings is taken and the value for each input channel is averaged over that 300 ms. The resulting output signal is smooth and positive in magnitude, which is easily processed by a control algorithm. As mentioned in the methods, the implemented control algorithm was the Kalman Filter. This algorithm necessitates a training and testing phase. The system can successfully recognize new sEMG data for three DOFs after a five-minute training period. However, 19 the device’s ability to recognize multiple different motions at the same time decreases as the number of motions increases, as seen through the RMSE results. By incorporating buttons on the outside of the 3D-printed case to activate the training and testing phases, the user does not need to worry about any of the software or hardware inside (Fig. 4G). They press the training button, follow the preprogrammed motions the prosthetic limb performs, and then take control of the hand. If they have previously trained the device using the same electrode configuration, they can press the testing button to skip retraining. There are two sets of buttons currently implemented: one set is a training for one DOF (opening and closing a fist), and the second set is for five DOF (thumb, index, middle, ring, and little flexion/extension). With the incorporated battery pack (Fig. 4F) and a belt loop on the casing, the system is completely portable. Fully assembled, the system has dimensions of 22.9 by 19.9 by 5.4 cm, and its proportion to an average-height male participant is shown in Figure 5. The prosthesis (Fig. 4E) needs its own power source. Using the LCS, participants were able to manipulate multiple degrees of freedom after training, as Figure 6 demonstrates. 20 Figure 4. An overview of the low-cost system. The system has dimensions of 22.9 by 19.87 by 5.4 cm. A) Snap lead electrodes. An imbedded electrode sleeve was developed as an alternative to snap electrodes based off of Harding et al.’s work, not pictured here [23] . B) Signals are acquired and filtered by the Muscle Spikershield Pro from Backyard Brains. C) Data is sampled at 1 kHz by the Arduino Mega microcontroller. D) The Raspberry Pi 3b+ smooths the data, holds/runs the control algorithm, and controls the prostheses. E) The prosthetic hand receives and actuates instructions from the Raspberry Pi (HANDI Hand shown here). F) This external battery provides 8+ hours of portable use. G) Operational buttons allow the user to run the training and testing algorithms with a button press [24]. Figure 5. The purpose of multiarticulate prostheses and proportional control systems is to allow amputees to intuitively complete activities of daily living. A participant is shown here wearing the portable LCS on his belt. The electrodes are connected to his right arm, and he is grasping a water bottle using an inexpensive prosthesis (HANDI Hand) [9], [24]. 21 Figure 6. A participant performs three different DOF after training: rest position (not counted as a DOF), index flexion, pinky flexion, and thumb flexion. The prosthesis shown here is another 3D-printed option (ADA Hand). 22 DISCUSSION The aim of this project was to design a low-cost system (LCS) for proportional prosthetic control. To determine that it had comparable performance to a high-cost system (HCS) (Grapevine Neural Interface Processor; Ripple Neuro LLC, Salt Lake City, UT USA), a signal-to-noise ratio (SNR) test and a root mean squared error (RMSE) test were conducted. Where the HCS costs around $64,000 USD total, the LCS cost around $675 USD in materials; both systems performed similarly when using six electrodes. The criteria outlined in Table 1 were all achieved. The signal-to-noise ratio (SNR) test aimed to determine whether the surface electromyographic (sEMG) signals recorded by the LCS were “clean” enough to correctly inform a control algorithm. A clean signal refers to low noise, which is important for the LCS. The LCS is restricted to six channels of sEMG, whereas other systems boast upwards of 32 [11], [15]. With so few channels, low signal quality (measured as SNR) can render control algorithms useless. For example, the large standard deviation in the HCS for the grasping motion shown in Figure 3A would indicate poor training results for the control algorithm. Fortunately, an exploratory number of tests (n = 3) yielded comparable values between the LCS and HCS (Fig. 3A). At most, the SNR of the LCS was 44% worse than the HCS for two of the motions we tested and was only at most 37% worse for a third motion (p’s < 0.05; TOST). The results of the TOST show the LCS’s acquisition capabilities are viable for a prosthetic control system. The second test sought to verify the LCS’s control performance against the HCS. As stated above, root-mean-squared error (RMSE) measurements give an idea of how well 23 the system can accurately translate sEMG activity to movement. The LCS and HCS performed similarly in these test conditions for intended and unintended movement RMSE. Intended RMSE indicates how well the algorithm predicts user input when the user attempts to control certain DOFs. Unintended motion refers to the algorithm predicting motion on a DOF when the user is not attempting to move that DOF. From the TOST, the LCS RMSE for one DOF was no worse than 6% than the HCS RMSE. At six DOFs, LCS intended RMSE was actually at least 4% better than the HCS RMSE (p’s < 0.05; TOST; Fig. 2B). LCS unintended RMSEs were statistically no greater than 0.03 than the HCS unintended RMSEs from one to six trained DOFs (p’s < 0.05, TOST; Fig. 2C) [35]. While not formally tested, participants were able to control a virtual hand when attached to the HCS with this setup, which is promising. After establishing reasonable user control, all LCS components were compacted into a portable package. A 3D-printed case houses all the components, including a battery to power the Raspberry Pi and Arduino for at least eight hours. A belt clip was attached to make the device wearable. The case is 23x20x13 cm (Fig. 5). The battery, components, and casing all constitute 2.30 pounds. Although the system is portable, it is not as compact as some systems on the market, such as Open Bionics’ Hero Arm. However, that arm costs approximately $3,000 USD, and only allows for control of one DOF at a time [5]. In order to switch between different DOFs, the user must manually press a button. The LCS system costs less even with a 3D-printed hand included (HANDI Hand: $550 USD [9]) and can be trained on multiple DOF for use without manually switching between DOFs. The system is limited by its components. The low-cost system’s function is most importantly restricted by its six-channel input through the Backyard Brains Spikershield 24 Pro. When control algorithms such as the Kalman Filter are fed more data, control quality increases. The Spikershield Pro also determines the cost of the system, as it is the most expensive component at $399 USD. It is also significant that the $675 USD cost of the LCS is the cost of materials only. Unlike the costs of other prosthetic devices and systems, it does not include other productions costs such as labor, regulatory fees, overhead, or support costs. The six-channel input directly affected the RMSE experiment results. Although the LCS and HCS results were comparable, the observed intended movement RMSEs for both systems were above 0.3 with only one and two trained DOF (Fig. 3B). This intended RMSE value is higher than RMSE values found in literature, which did not exceed 0.3 with six trained DOF [15]. George et. al used the same HCS in that study as we did in this one. The difference between the literature and observed RMSEs is the number of sEMG channels used. The HCS is meant to take in 32 channels of EMG data. It was constrained to six channels to better compare with the LCS. This indicates that the system requires more channels of input data to improve performance. The Kalman Filter control algorithm also has restrictions. It assumes linear correlation between sEMG activity and exact hand position. This linearity presents a problem because a human hand does not necessarily need to maintain constant muscular exertion to maintain a position. The action of making a fist takes effort while keeping your hand closed once in position does not. In addition to this exertion issue, is the problem of unwanted movement. When an intact individual flexes one finger, that is the only digit that moves (excluding tendon-caused residual motion). This fine control allows humans to manipulate small objects, type, play games, etc. However, if an amputee attempting the 25 same movement of one finger unintentionally moves other digits, the previously mentioned activities become impossible. These issues with the Kalman Filter are typical of proportional control algorithms [7]. There has been work done by the University of Utah Center for Neural Interfaces as well as other institutions to mitigate this unintentional movement [7], [15], but those methods would also increase the computation time of the low-cost system. Another limitation of the design is the portability problems mentioned previously. The system is rather bulky and will need time and innovation to compact. The tests conducted contain their own assumptions and restrictions. The largest restriction is the relatively small sample size (N = 3 for SNR and N = 6 for RMSE). On top of this, all participants were intact (i.e., non-amputees). Therefore, it is difficult to make any broad claims of performance, especially with amputees. Apart from population, the RMSE test was also conducted offline, meaning a MATLAB version of the Kalman Filter algorithm on the Raspberry Pi was used to calculate RMSEs. The test assumes there are no differences between functionality between the python Kalman Filter on the Pi and the MATLAB analog on a separate computer. This limitation exists due to the inability of the LCS to record intended motion and actual Kalman Filter output at the same time. Another limitation of the RMSE experiment is that the sEMG data were not collected on both systems simultaneously due to differences in recording and analysis between the LCS and HCS. This means that although the training sets of intended motions were identical between the two systems, the data itself was not directly comparable between the two. Perhaps a participant did well for the high-cost training but poor on the low-cost. This test could not reasonably account for that case. 26 Despite the aforementioned limitations, this project shows that cost reduction in this field is possible. The LCS marries accessible technology with research-grade control processes to provide a control system comparable to a research-grade system. The LCS can functionally control 3+ DOFs and is completely portable. Although $675 USD was the cost of materials alone, the project would ideally be open source, allowing researchers and eventually amputees access. By being open source, any interested party would only need to gain access to the materials necessary and download the software for the LCS. Currently, the LCS does not stand a suitable replacement for market options such as the Hero Arm or Ripple Nomad. However, researchers can improve the system further to become a fully viable option for amputees. Current points of improvement include replacing the expensive sEMG acquisition device (Backyard Brains Spikershield Pro) with a less costly alternative. Such a project is already underway in the University of Utah Center for Neural Interfaces. The new device could increase the EMG channels available from six to eight and might include Bluetooth functionality (ADS1298; Texas Instruments, Dallas, USA). This would increase the data available to the system (potentially without reducing runtime) and improve portability by reducing space and wiring concerns. Another current focus for improvement is interfacing multiarticulate prostheses such as the TASKA hand [25] or the DEKA Luke Arm [26] with the LCS. Providing the system support with more robust hands would allow researchers to conduct true functional tests. These tests would potentially involve amputees or intact participants conducting acts of daily living (opening doors, picking up objects, etc.) and comparing the LCS performance to an HCS. 27 If this system or aspects of it can be used successfully with amputee participants, the technology could have significant global impact. The exorbitant cost of healthcare is an issue facing the entire world, so work to reduce barriers in the field are incredibly important. This project is especially salient to developing nations, which house roughly 40 million amputees and whom only 5% have access to prosthetic care [36]. In the best case the work conducted in this project will provide high-quality technology to someone that the current market leaves out. 28 ACKNOWLEDGEMENT This work was funded by: The Undergraduate Research Opportunities Program, DARPA, BTO, Hand Proprioception and Touch Interfaces program, Space and Naval Warfare Systems Center, Pacific, Contract No. N66001-15-C-4017; NSF Award No. ECCS-1533649; NSF Award No. CHS-1901492 and NSF Award No. GRFP-1747505. 29 REFERENCES [1] G. McGimpsey and T. Bradford, “Limb Prosthetics Services and Devices,” Bioeng. Inst. Cent. Neuroprosthetics Worcest. Polytech. Inst., pp. 1–35. [2] D. K. Blough, S. Hubbard, L. V. McFarland, D. G. . Smith, J. M. Gambel, and G. E. Reiber, “Prosthetic cost projections for servicemembers with major limb loss from Vietnam and OIF/OEF,” J. Rehabil. Res. Dev., vol. 47, no. 4, p. 387, 2010, doi: 10.1682/JRRD.2009.04.0037. [3] E. Pröbsting et al., “Ease of Activities of Daily Living with Conventional and Multigrip Myoelectric Hands,” JPO J. Prosthet. Orthot., vol. 27, no. 2, pp. 46–52, Apr. 2015, doi: 10.1097/JPO.0000000000000058. [4] I. Ku, G. K. Lee, C. Y. Park, J. Lee, and E. Jeong, “Clinical outcomes of a low-cost single-channel myoelectric-interface three-dimensional hand prosthesis,” Arch. Plast. Surg., vol. 46, no. 4, pp. 303–310, Jul. 2019, doi: 10.5999/aps.2018.01375. [5] “Hero Arm - User Guide,” Open Bionics. https://openbionics.com/hero-arm-userguide/ (accessed Jan. 20, 2020). [6] H. H. Sears and J. Shaperman, “Proportional myoelectric hand control: an evaluation,” Am. J. Phys. Med. Rehabil., vol. 70, no. 1, pp. 20–28, Feb. 1991, doi: 10.1097/00002060-199102000-00005. [7] E. Scheme and K. Englehart, “Training Strategies for Mitigating the Effect of Proportional Control on Classification in Pattern Recognition Based Myoelectric Control,” J. Prosthet. Orthot. JPO, vol. 25, no. 2, pp. 76–83, Apr. 2013, doi: 10.1097/JPO.0b013e318289950b. 30 [8] M. Ison and P. K. Artemiadis, “Proportional Myoelectric Control of Robots: Muscle Synergy Development Drives Performance Enhancement, Retainment, and Generalization,” IEEE Trans. Robot., 2015, doi: 10.1109/TRO.2015.2395731. [9] D. J. A. Brenneis, M. R. Dawson, and P. M. Pilarski, “Development of the HANDI Hand: An Inexpensive, Multi-Articulating, Sensorized Hand for Machine Learning Research in Myoelectric Control,” in Myoelectric Controls and Upper Limb Prosthetics Symposium, Fredericton, New Brunswick, Canada, Aug. 2017, p. 4. [10] N. Sreenivasan, D. F. Ulloa Gutierrez, P. Bifulco, M. Cesarelli, U. Gunawardana, and G. D. Gargiulo, “Towards Ultra Low-Cost Myoactivated Prostheses,” BioMed Res. Int., vol. 2018, p. 9634184, 2018, doi: 10.1155/2018/9634184. [11] J. A. George, T. S. Davis, M. R. Brinton, and G. A. Clark, “Intuitive neuromyoelectric control of a dexterous bionic arm using a modified Kalman filter,” J. Neurosci. Methods, vol. 330, p. 108462, Jan. 2020, doi: 10.1016/j.jneumeth.2019.108462. [12] “Development of a portable take-home system for control of advanced prostheses.” https://www.abstractsonline.com/pp8/#!/4649/presentation/30647 (accessed Dec. 04, 2019). [13] “Grapevine Processors – Ripple Neuro.” https://rippleneuro.com/rippleproducts/grapevine-processors/ (accessed Dec. 04, 2019). [14] D. M. Page et al., “Motor Control and Sensory Feedback Enhance Prosthesis Embodiment and Reduce Phantom Pain After Long-Term Hand Amputation,” Front. Hum. Neurosci., vol. 12, 2018, doi: 10.3389/fnhum.2018.00352. 31 [15] J. A. George, M. R. Brinton, C. C. Duncan, D. T. Hutchinson, and G. A. Clark, “Improved Training Paradigms and Motor-decode Algorithms: Results from Intact Individuals and a Recent Transradial Amputee with Prior Complex Regional Pain Syndrome,” in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jul. 2018, pp. 3782–3787, doi: 10.1109/EMBC.2018.8513342. [16] “Prosthetic Coverage.” https://www.medicare.gov/coverage/prosthetic-devices (accessed Dec. 04, 2019). [17] E. Biddiss and T. Chau, “Upper-Limb Prosthetics: Critical Factors in Device Abandonment,” Am. J. Phys. Med. Rehabil., vol. 86, no. 12, p. 977, Dec. 2007, doi: 10.1097/PHM.0b013e3181587f6c. [18] “Teach, Learn, and Make with Raspberry Pi – Raspberry Pi.” https://www.raspberrypi.org (accessed Dec. 04, 2019). [19] “Arduino Mega 2560 Rev3.” https://store.arduino.cc/usa/mega-2560-r3 (accessed Dec. 04, 2019). [20] A. K. Vaskov et al., “Cortical Decoding of Individual Finger Group Motions Using ReFIT Kalman Filter,” Front. Neurosci., vol. 12, 2018, doi: 10.3389/fnins.2018.00751. [21] J. Nieveen et al., “Polynomial Kalman filter for myoelectric prosthetics using efficient kernel ridge regression,” in 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER), May 2017, pp. 432–435, doi: 10.1109/NER.2017.8008382. 32 [22] A. J. Thurston, “Paré and prosthetics: the early history of artificial limbs,” ANZ J. Surg., vol. 77, no. 12, pp. 1114–1119, Dec. 2007, doi: 10.1111/j.14452197.2007.04330.x. [23] K. J. Zuo and J. L. Olson, “The evolution of functional hand replacement: From iron prostheses to hand transplantation,” Plast. Surg., vol. 22, no. 1, pp. 44–51, 2014. [24] C. K. Battye, A. Nightingale, and J. Whillis, “The use of myo-electric currents in the operation of prostheses,” J. Bone Joint Surg. Br., vol. 37-B, no. 3, pp. 506–510, Aug. 1955, doi: 10.1302/0301-620X.37B3.506. [25] “TASKA Prosthetics | Doing More Builds Confidence.” http://www.taskaprosthetics.com/ (accessed Jan. 21, 2020). [26] L. Resnik, S. L. Klinger, and K. Etter, “The DEKA Arm: Its features, functionality, and evolution during the Veterans Affairs Study to optimize the DEKA Arm,” Prosthet. Orthot. Int., vol. 38, no. 6, pp. 492–504, Dec. 2014, doi: 10.1177/0309364613506913. [27] D. J. Atkins, D. C. Y. Heard, and W. H. Donovan, “Epidemiologic Overview of Individuals with Upper-Limb Loss and Their Reported Research Priorities,” JPO J. Prosthet. Orthot., vol. 8, no. 1, pp. 2–11, Winter 1996. [28] E. Biddiss, D. Beaton, and T. Chau, “Consumer design priorities for upper limb prosthetics,” Disabil. Rehabil. Assist. Technol., vol. 2, no. 6, pp. 346–357, Jan. 2007, doi: 10.1080/17483100701714733. [29] R. Eveleth, “When State-of-the-Art is Second Best,” NOVA Tech & Engineering. [30] R. Stopforth, D. Riet, G. Bright, and O. Diegel, “A Low Cost Design of a 3D Printed Multi-fingered Myoelectric Prosthetic Hand,” 2015, pp. 85–117. 33 [31] C. Jeffrey, “Open Bionics’ 3D-printed ‘Hero Arm’ is now available in the US TechSpot.” https://www.techspot.com/news/79515-open-bionics-3d-printed-heroarm-now-available.html (accessed Mar. 02, 2020). [32] M. Atzori, M. Cognolato, and H. Müller, “Deep Learning with Convolutional Neural Networks Applied to Electromyography Data: A Resource for the Classification of Movements for Prosthetic Hands,” Front. Neurorobotics, vol. 10, 2016, doi: 10.3389/fnbot.2016.00009. [33] J. L. Rogers, K. I. Howard, and J. T. Vessey, “Using significance tests to evaluate equivalence between two experimental groups,” Psychol. Bull., vol. 113, no. 3, pp. 553–565, 1993, doi: 10.1037/0033-2909.113.3.553. [34] A. N. Harding et al., “Undergraduate Research Journal,” p. 1. [35] J. A. George, S. Radhakrishnan, M. R. Brinton, and G. A. Clark, “Inexpensive and Portable System for Dexterous High-Density Myoelectric Control of Multiarticulate Prostheses,” presented at the IEEE Engineering in Medicine and Biology Society [submitted], Montréal, Québec, Canada. [36] M. Marino et al., “Access to prosthetic devices in developing countries: Pathways and challenges,” in 2015 IEEE Global Humanitarian Technology Conference (GHTC), Oct. 2015, pp. 45–51, doi: 10.1109/GHTC.2015.7343953. 34