Design of an exoskeletal human motion capture system (SenSuit™)

Human motion capture has a wide variety of applications in the entertainment and medical industries. Actors using motion capture devices provide realistic motion inputs for cartoons, virtual reality environments™, and computer and robot animation, resulting in tremendous time and cost savings. Medical applications include range of motion studies to diagnose injuries or identify insurance fraud, biomechanics studies of human performance and calculation of joint stresses, and ergonomics studies of humans in the workplace. There are common problems facing all methods of motion capture: how to attach the device to the individual's limbs, what sensors to use and how to use them, how to transmit data and convert it into a usable form, calibration of the device, data display, user comfort, and device reliability. Even when these problems are addressed, there are limitations in the kinematic model as well as human joint anomalies that make all methods imperfect. Currently, there are optical, magnetic, and exoskeletal devices for motion capture that vary widely in terms of performance, cost and limitations. Considering the likely environment and performance needs of the Sarcos Research Corporation, the SenSuit™ was built as an exoskeletal device. Creation of the SenSuit™ involved overcoming three major hurdles: the soft tissue interface, accurate joint angle measurement, and sensor design. The soft tissue interface is the series of rigid plates that are placed on skeletal landmarks located near the surface of the user's skin. Through appropriate location of the plates, a consistent, stable fit to the skeleton was achieved for users, which enhanced joint angle data. Accurate joint angle measurements were achieved either by aligning sensor rotation centers with approximate joint rotation centers or by computationally transforming the outputs of three degree of freedom sensor clusters located to reduce nonlinearities. A software routine allowed for quick, linear calibration of the individual. Joint angle sensors were designed that were small, linear, robust, and resistant to wear and contaminants. The SenSuit™ has proven itself both comfortable and reliable. It has been thoroughly tested in real-world applications, including real-time driving of graphical and robotic figures, as well as the programming of various robotic figures.

DESIGN OF AN EXOSKELETAL HUMAN MOTION CAPTURE SYSTEM (SENSUIT™) by James Kolozs A thesis submitted to the faculty of The University of Utah in partial fulfillment of the requirements for the degree of Master of Science Department of Bioengineering The University of Utah June 1998 Copyright © James Kolozs 1998 All Rights Reserved THE U N I V E R S I T Y OF UTAH G R A D U A T E SCHOOL S U P E R V I S O R Y C O M M I T T E E A P P R O V A L of a thesis submitted by James K o l o z s This thesis has been read by each member of the following supervisory committee and by majority vote has been found to be satisfactory. T H E U N I V E R S I T Y OF UTAH G R A D U A T E S C H O O L F I N A L R E A D I N G A P P R O V A L To the Graduate Council of the University of Utah: I have read the thesis of James K o l o z s in its final form and have found that (1) its format, citations, and bibliographic style are consistent and acceptable; (2) its illustrative materials including figures, tables, and charts are in place; and (3) the final manuscript is satisfactory to the supervisory committee and is ready for submission to The Graduate School. Chair, Tn C. J a c o b s en jpervisory Committee Approved for the Major Department K a r i n D. C a l d w e l l Chair/Dean Approved for the Graduate Council Ann W. Hart Dean of The Graduate School A B S T R A C T Human motion capture has a wide variety of applications in the entertainment and medical industries. Actors using motion capture devices provide realistic motion inputs for cartoons, virtual reality environments™, and computer and robot animation, resulting in tremendous time and cost savings. Medical applications include range of motion studies to diagnose injuries or identify insurance fraud, biomechanics studies of human performance and calculation of joint stresses, and ergonomics studies of humans in the workplace. There are common problems facing all methods of motion capture: how to attach the device to the individual's limbs, what sensors to use and how to use them, how to transmit data and convert it into a usable form, calibration of the device, data display, user comfort, and device reliability. Even when these problems are addressed, there are limitations in the kinematic model as well as human joint anomalies that make all methods imperfect. Currently, there are optical, magnetic, and exoskeletal devices for motion capture that vary widely in terms of performance, cost and limitations. Considering the likely environment and performance needs of the Sarcos Research Corporation, the SenSuit™ was built as an exoskeletal device. Creation of the SenSuit™ involved overcoming three major hurdles: the soft tissue interface, accurate joint angle measurement, and sensor design. The soft tissue interface is the series of rigid plates that are placed on skeletal landmarks located near the surface of the user's skin. Through appropriate location of the plates, a consistent, stable fit to the skeleton was achieved for users, which enhanced joint angle data. Accurate joint angle measurements were achieved either by aligning sensor rotation centers with approximate joint rotation centers or by computationally transforming the outputs of three degree of freedom sensor clusters located to reduce nonlinearities. A software routine allowed for quick, linear calibration of the individual. Joint angle sensors were designed that were small, linear, robust, and resistant to wear and contaminants. The SenSuit™ has proven itself both comfortable and reliable. It has been thoroughly tested in real-world applications, including real-time driving of graphical and robotic figures, as well as the programming of various robotic figures. v T A B L E O F C O N T E N T S ABSTRACT i LIST OF TABLES vi LIST OF FIGURES i ACKNOWLEDGEMENTS xi 1. INTRODUCTION 2. DEFINITIONS 3. BACKGROUND 3.1 Technical Challenges in Motion Capture 3.2 Limitations of Motion Capture 1 3.3 Current Motion Capture Technologies 1 3.4 SenSuit™ 1 4. SOFT TISSUE INTERFACE 2 4.1 Approach 2 4.2 Plate Material 2 4.3 Design 3 4.4 Summary 4 5. SENSORS 5 5.1 Motivation 5 5.2 Theory 5.3 General Design 5.4 Performance 5.5 Summary 6. JOINT ANGLE MEASUREMENT 6.1 Approach 6.2 Head/Neck 6.3 Elbow/Knee 6.4 Wrist 6.5 Waist 6.6 Three DOF Clusters 6.7 Summary 7. ELECTRONICS/SOFTWARE 7.1 SenSuit™ Electronics 7.2 Calibration Procedures 7.3 Shoulder Transformation Algorithm 7.4 Waist Transformation Algorithm 138 7.5 Visualization Program 141 7.6 Summary 142 8. SENSUIT™ PERFORMANCE 144 8.1 DOFs and ROMs 144 8.2 SenSuit™ Components and Comfort 145 8.3 Transformation Algorithms 148 8.4 Accuracy 159 8.5 Repeatability and Drift 161 8.6 Summary 164 9. CONCLUSIONS 165 10. SUGGESTIONS FOR FUTURE RESEARCH 167 11. REFERENCES 170 vii L I S T O F T A B L E S Table Page 3-1. The joints and corresponding DOFs measured in the current SenSuit™ 23 4-1. Selected material properties of SenSuit™ plates 29 5-1. Linear Hall effect sensor selected characteristics 55 5-2. Statistics for the HallJoint noise test 60 5-3. Strengths of common magnetic fields 61 7-1. Calculated nonlinearity estimates for the three DOF ball joints 106 7-2. The DH parameters 117 7-3. The nominal DH parameters 118 8-1. SenSuit™ DOFs and associated rotation axes and ROMs 145 8-2. SenSuit™ component weights 147 8-3. Flexion and abduction coupling during pure axial rotation of the right arm 155 8-4. Flexion and abduction coupling during pure axial rotation of the left arm 156 8-5. Flexion and abduction coupling during pure axial rotation of the waist 159 8-6. Accuracy results for selected DOFs 161 8-7. Repeatability results for the left shoulder 163 8-8. Drift results for the left shoulder 163 L I S T O F F I G U R E S Figure Page 1-1. The fourth version SRC SenSuit™ 3 2-1. Human reference position for joint motion conventions 5 3-1. Typical optical system setup 12 3-2. Typical magnetic system setup 15 3-3. Typical exoskeletal system setup 17 3-4. Previous versions of the SRC SenSuit™ 20 3-5. Current (fourth) version of the SRC SenSuit™ 22 4-1. Bony protrusions used to guide the placement of SenSuit™ plates 26 4-2. The general structure of current SenSuit™ plates 29 4-3. Previous SenSuit™ torso plate 31 4-4. Current torso plates 32 4-5. Baseball style hat used in previous SenSuits™ 34 4-6. Current head attachment 34 4-7. Shoulder DOFs and ROMs 35 4-8. Failed shoulder soft tissue interface (third SenSuit™) 36 4-9. Current shoulder attachment 37 4-10. Previous elbow plate and associated linkages 39 4-11. Current elbow attachment 40 4-12. Current wrist attachment 41 4-13. Previous hip attachment 43 4-14. Current hip attachment 44 4-15. Mechanisms to hold the hip assembly in place 45 4-16. Previous SenSuit™ knee attachment 46 4-17. Current knee attachment 47 4-18. Previous foot attachment 48 4-19. Current foot attachment 49 5-1. The Hall effect 54 5-2. Prior SRC rotary displacement transducers using Hall effect sensors 55 5-3. The Hall Joint 56 5-4. Interpreting angular changes from a HallJoint 57 5-5. HallJoint linearity 59 5-6. Strength of the magnetic field along the axis of the test magnet 62 5-7. Comparison of sensor sizes 63 5-9. ROM variations of the HallJoint 64 5-10. An extended shaft HallJoint for measuring axial rotation 65 5-11. The three DOF HallJoint 66 5-12. The reinforced HallJoint 67 5-13. HallJoint static strength test and failure mode 68 6-1. The cervical vertebral column 72 6-2. Head DOF measurement 74 6-3. The elbow and knee joints 75 6-4. Elbow DOF measurement 76 6-5. Knee DOF measurement 77 6-6. Wrist CORs 78 6-7. Joints in the wrist 79 6-8. Wrist DOF measurement 80 6-9. The vertebral column 81 6-10. Third SenSuit™ waist measurement 82 6-11. Failed three DOF waist sensor viewed from side 83 x 6-12. Waist DOF measurement for the current SenSuit™ 84 6-13. Shoulder CORs 85 6-14. The shoulder joint 86 6-15. Shoulder ICORs of the head of the right humerus 87 6-16. Third SenSuit™ shoulder mechanism 88 6-17. Shoulder DOF measurement 90 6-18. Hip CORs 91 6-19. The hip joint 92 6-20. Hip DOF measurement 93 6-21. Foot CORs 94 6-22. Bones in the ankle and foot 94 6-23. Ankle and foot DOF measurement 96 7-1. E-Box location on the SenSuit™ 98 7-2. Dataflow through the E-Box 99 7-3. The difference between the limb angle and sensor angle 101 7-4. Dataflow and setup for controlling a robot figure 102 7-5. Example of sensor coupling 105 7-6. Relation of variables to the SenSuit™ and arm 108 7-7. Calculating the slider orientation v 110 7-8. Calculating the slider length s 112 7-9. The DH parameters and the manipulator in the nominal position 117 7-10. The DH parameters and manipulator in the zero position 118 7-11. The goal of the inverse kinematics and relevant variables 122 7-12. Calculating the flexion angle 122 7-13. Calculating the abduction angle 123 7-14. Induced rotation: pure flexion 126 7-15. Induced rotation: pure abduction 127 xi 7-16. Induced rotation: p near z axis 127 7-17. The length of p in the z-v plane {Lk) 133 7-18. Calculating the value of 8k 133 7-19. The induced rotation for all values of flexion and abduction 135 7-20. Induced rotation versus abduction for constant values of flexion 136 7-21. Induced rotation versus flexion for constant values of abduction 137 7-22. Coupling of waist flexion and abduction to waist rotation 140 7-23. SRC visualization program 141 7-24. The Drillis and Contini model (33) 142 8-1. SenSuit™ DOFs 146 8-2. Axial rotation as a function of time for the first right arm trial 149 8-3. Transformation algorithm effect on left arm flexion coupling 151 8-4. Transformation algorithm effect on left arm abduction coupling. 152 8-5. Transformation algorithm effect on right arm flexion coupling 153 8-6. Transformation algorithm effect on right arm abduction coupling 154 8-7. Right arm abduction coupling without algorithm 155 8-8. Transformation algorithm effect on waist flexion coupling 157 8-9. Transformation algorithm effect on waist abduction coupling 158 xii A C K N O W L E D G E M E N T S For this project, I received technical assistance from a number of people. Steve Jacobsen was my primary advisor throughout the project and was instrumental in the design of the new sensors and the shoulder soft tissue interface. James Hyde supervised me throughout the project and gave me feedback and ideas on improving the SenSuit™. Fraser Smith provided design feedback and suggestions. Todd Johnson advised me in the wiring of the SenSuit™ and performed modifications to the electronics box to make it compatible with the new SenSuit™. Tom Andrew helped design the carbon fiber plates, and his company, Ability Prosthetics, manufactured the plates. Dave Meer rewrote and improved much of the visualization program and provided computer assistance. Curtis Bowler wired the SenSuit™ together and advised me on efficient and reliable wiring design. Jeff Barrett helped with prototyping and machined many of the parts for the SenSuit™. Shane Olsen assisted me in the completion of the upper torso sections and helped design the sliders and waist sensors. John Hollerbach taught me the mathematics of robotics and helped me understand the problems encountered in writing the transformation algorithms. The current SenSuit™ is the fourth version created at the Sarcos Research Corporation. Many workers helped create the previous three versions, including Steve Jacobsen, Todd Johnson, Fraser Smith, Jed Marti, James Hyde, Eric Gundersen, Dave Knutti, Clark Davis, Todd Rytting, Nathan Mead, Dwight Potter, Wayco Scroggin, Rick Garrett. Keith Brendly, and others. Their work laid the foundation for the current SenSuit™, having dealt with many of the problems and design issues involved in creating an exoskeleton. I thank my committee, Steve Jacobsen, James Hyde, and Rick Rabbitt, for advising me throughout this project. I would also like to thank the Sarcos Research Corporation and the Center for Engineering Design at the University of Utah for their support. Not only did they fund this project, but they also provided facilities for the design, construction, and testing of the SenSuit™. Finally, I want to thank my family and friends for supporting and encouraging me in this endeavor. Special thanks to Deniz for her help throughout the project, especially with the presentation. xiv 1 . I N T R O D U C T I O N The past 5-10 years have seen the emergence of human, whole body motion capture devices. Using a variety of sensing technologies, these devices seek to transform the actual human movement of a subject into usable position, orientation, or joint angle data. Medical applications include range of motion studies, gait analysis, and joint force/torque estimation. The entertainment industry has shown great interest in whole body motion capture, primarily because it provides a simple and fast way to acquire realistic human motion for computer animation. Traditional computer animation and robot programming require that trajectories be designed for each individual degree of freedom. These are assembled to create the gross motion of the figure. This method requires an experienced animator and a tremendous time investment, since the animation is created essentially by hand. Motion capture allows the programming of all the degrees of freedom simultaneously, often in real-time. Real-time driving of a computer or robot figure is extremely difficult without motion capture. Motion capture data are inherently natural and human looking, since the data are taken directly off the performer's body. Motion capture has been used to program robots and computer animations for cartoons, commercials, and movies. The sponsors of this project, the Sarcos Research Corporation (SRC) and the Center for Engineering Design (at the University of Utah), have created high performance, anthropomorphic, entertainment robots. They needed a whole body motion capture device to drive these robots in real-time (for a live interactive setting) or to program for autonomous entertainment displays. Commercially available optical and magnetic motion capture systems were considered but were deemed unsuitable for SRCs 2 needs and working environment. Thus, the decision was made to build an exoskeletal, whole body motion capture system, the SenSuit™. Three SenSuits™ were built, each an improvement on the last. However, many problems continued to plague the SenSuit™, including the following: 1. Unstable soft tissue interface 2. Inaccurate joint angle measurement 3. Large and unreliable sensors 4. Poor user comfort because of weight, bulk, and design 5. Poor reliability due to plate, joint, and wire breakage This thesis details the design of a new, fourth version SRC SenSuit™, shown in Figure 1-1. The primary contribution of this thesis are as follows. The soft tissue interface was extensively revised to provide a consistent, stable fit on the user. New joint angle sensors were designed that were small, robust, and accurate. The sensors and soft tissue interface were arranged in order to obtain the most accurate joint angle measurement. Stronger, lighter materials were used throughout the SenSuit™, along with designs intended to make the SenSuit™ both more comfortable and more reliable. Algorithms were developed to decouple the sensors of the shoulder and waist. Finally, tests were performed on the SenSuit™ to determine the accuracy, repeatability, and drift of selected joints. Figure 1-1. The fourth version SRC SenSuit™. 2 . D E F I N I T I O N S This chapter defines the common terms and acronyms used throughout this thesis. Also, the naming conventions that describe human limb movement, which differ slightly from common medical terminology, are explained below. • 2D/3D - Two/Three-Dimensional • Avatar - A computer generated graphical figure that follows SenSuit™ commands. • COR - Center of Rotation • DOF - Degree of Freedom • HallJoint - The type of sensor used in the majority of SenSuit™ joints. • ICOR - Instantaneous Center of Rotation • Performer/User -- The person wearing the motion capture device. • ROM --Range of Motion • SRC - Sarcos Research Corporation • Workspace - The volume of space that a limb(s) can occupy through the various movements of its joints. Figure 2-1 shows the reference position of a human for the purpose of naming joint movements. The convention used here combines the normal medical definitions of joint movements with the consistency of a coordinate system. All movements are named rigorously based on the definitions below and can be assigned a rotation axis. For instance, elbow motion is a flexion or extension, occurring about the x axis. This convention does rename some medical definitions of motion (such as wrist Figure 2-1. Human reference position for joint motion conventions. Adapted from (1). supination/pronation to wrist rotation), but it has the benefit of a consistent naming system. Also, by assigning motions to coordinate system axes, it simplifies the construction of the computer model. The following conventions are used in describing the movement of a joint degree of freedom (DOF) when the person stands in the anatomical reference position: • Flexion/Extension - Movement in the sagittal plane of the body or rotation about the x axis. • Abduction/Adduction - Movement in the frontal plane of the body or rotation about the y axis. • Rotation - Rotation about the vertical (z) axis. 3 . B A C K G R O U N D 3 . 1 T e c h n i c a l C h a l l e n g e s in M o t i o n C a p t u r e The main goal of human motion capture is to track the motions of the limbs reliably and accurately. Some systems track the positions of a set of predefined points and infer limb position and orientation. Other systems can measure the position and orientiation of a sensor simultaneously and thus location and orientation of a limb. Still others measure the joint angles directly with sensors mounted across the anatomical joints. Regardless of the method, they all share common problems that must be addressed. 3 . 1 . 1 Attachment All current motion capture systems need to attach markers or sensors of some type to the performer's body. Several choices must be made: where to attach the devices, how to attach them, and what materials to use. Ideally, the movement of the skeleton is measured; however, since it is covered by soft tissue, placement of a sensor does not guarantee accurate measurement of the underlying skeletal motion. The attachment interface should be usable by a large population and fit in a comfortable and consistent manner. The interface should be stable so that the sensors do not wander after prolonged use. 3 . 1 . 2 S e n s o rs Motion capture systems usually measure the three-dimensional (3D) postions of points in space, the 3D position and 3D orientation of sensors, or joint angles. Regardless of the parameter measured, it is important to have a sensor with adequate resolution and 8 accuracy. Normally, the sensors required for these various types of measurement are not readily or widely available, and systems usually incorporate custom sensors. 3 . 1 . 3 Data Transmission If sensors are mounted on the body, their data must be transmitted to a dedicated processor or computer either by tether or wireless communication. Tethered systems can be parallel or serial. If parallel, the wires from each sensor must leave the person and connect to a processor or computer. Parallel tethers tend to be bulky, since the tether contains several wires per sensor. Serial tethers use an electronics package worn by the user to receive all the sensor data, and then output the multiplexed data to the external processor or computer via a serial line. Although this considerably reduces the number of wires in the tether, it requires a compact set of multiplexers and analog to digital converters in the electronics package worn by the user. Wireless methods include radio frequency and infrared. All wireless methods require that batteries, electronics, and transmitters/receivers be worn on the user. 3 . 1 . 4 Data Transformation Once the sensor data arrive at the computer, it is not necessarily ready for use. The data may be transformed in a variety of ways before being output to a kinematic model. The data may be filtered to reduce noise. The software may compensate for sensor nonlinearities. Two-dimensional (2D) data from cameras need to be converted into 3D positional data. Position and orientation data are often converted into joint angle data. Magnetic systems can compensate for environmental distortions in their fields by the use of a spatial lookup table. Regardless of the system, computer processing of the data is usually required. 9 3 . 1 . 5 Calibration All motion capture systems must be calibrated to ensure the best estimation of various parameters. Depending on the system, calibration can be used to determine camera coordinate systems, magnetic field anomalies in the environment, sensor gains and offsets, and other parameters. Calibration is advisable whenever the motion capture environment changes. 3 . 1 . 6 Display Data that have been conditioned through transformation and calibration are ready to be used. Much of motion capture uses a graphical format for the output, such as an icon of a virtual person, thus requiring the creation of a computer model. Simpler displays are the raw data or joint angle traces - tracking the movement of a single joint in time. Robots can also be driven by motion capture data, and in these cases the robot becomes the display. 3 . 1 . 7 Comfort Although it seems like a minor point, the comfort of the user is important, since the goal is human motion capture. A system that is uncomfortable or difficult to use will be avoided. Unfortunately, there seems to be a tradeoff between the comfort of the user and the reliability of the data: The more uncomfortable the user (through tightly attached sensors), the better the resulting data, since extraneous motions have been reduced. 3 . 1 . 8 Reliability Reliability is also a key issue and must be built into the system at all levels. The less time that is spent dealing with repairs, glitches, or computer crashes, the more that can be spent acquiring usable data. Especially in priority projects, this lost time becomes lost revenue. 10 3 . 2 L i m i t a t i o n s of M o t i o n C a p t u r e Even when a motion capture system is working perfectly, there are still inherent limitations to the accuracy of the data. All systems suffer from these limitations, in addition to those specific to their technologies. 3 . 2 . 1 Kinematic Model A model is an approximation, and the kinematic model used by a motion capture system for displaying or quantifying motion is an approximation of the performer. The motions of humans are subtle, and there are many DOFs that are either difficult to measure or must be approximated by simpler DOFs (such as the spine). These "missing joints" constitute omitted motion in the model. The performer's link lengths need to be measured and utilized by the model. Since human joints are hidden within the body, it is difficult to get an accurate measurement of the link lengths. The accuracy of the model is also limited by the accuracy of any calibrations that were performed. The workspace of the model is limited by the range of motion (ROM) of its joints. For robots, this is determined by mechanical stops, and in software, limits may be programmed. Regardless, it may be possible for the performer to move a limb outside the ROM of the model and lose registration between the user and the model. When this happens, it appears that the sensors have stopped working. When the limb is brought back into the model workspace, registration is reacquired. Finally, the performer and model may have different boundaries around their respective skeletons, which makes it possible to intersect limbs on the model when the performer's limbs are brought close together. i 1 3 . 2 . 2 Human Joint Anomalies All motion capture systems are limited by the human bodies they are trying to track. Humans have evolved joints and do not have the engineered perfection assumed by most models. Namely, many joints have instantaneous CORs that move depending on the angle of the limb. Instead of being simple hinges, many joints slide as well as rotate. Sometimes the CORs move through an arc, as in the knee, or can even jump discontinuously within the joint, as in shoulder abduction (2, 3). These movements subtly change joint offsets and link lengths. Further, the focus of motion capture is the motion of the skeleton. However, the skeleton is entirely covered by varying amounts of soft tissues. This makes it difficult to separate the motion of the skeleton from the overlying tissues. 3 . 3 C u r r e n t M o t i o n C a p t u r e T e c h n o l o g i e s With the increases in computing power and the sophistication of computer algorithms, it is foreseeable that future motion capture systems might incorporate cameras and software to scan the body in real-time without the use of special sensors. Body scanning is possible today, but it is extremely slow and expensive. Cyberware makes a system that takes about 15 s to scan the body, and it costs over $400,000 (4). This is far from real-time and very costly compared to current motion capture systems. However, we estimate that real-time body scanning is about 10-15 years away. To satisfy today's needs for motion capture, several systems have been developed. The most prominent are optical, magnetic, and exoskeletal. 3 . 3 . 1 Optical Optical systems rely on light reflected or emitted from markers on the performer and viewed by specialized cameras. Data from the cameras are then processed to produce 3D coordinates of each marker. A typical optical system is shown in Figure 3-1. Both 13 Motion Analysis Corporation and Adaptive Optics Associates use this type of setup (5, 6). In this type of system, the user wears a body suit onto which reflective balls are attached. Special cameras placed in the environment have unique views of the reflectors. A video processor calculates the two dimensional (2D) coordinates of the markers for each camera and transmits these data to a computer. The computer then solves for the 3D coordinates of the markers. These types of systems are not real-time. An alternative optical approach is that taken by Northern Digital. Their Optotrak system uses infrared LEDs as active markers placed on the body. Each marker is strobed sequentially to give the markers unique identities. Special position tracking units (a set of calibrated cameras) view the infrared marker signals, and a dedicated processor then finds the 3D coordinates of each marker in real-time. (7) Advantages of optical systems include having markers that are small and unobtrusive, allowing for virtually unrestricted movements of the performer. These systems can also be very accurate. For instance, the Optotrak can measure marker position accurately to less than a millimeter at a distance of 6 m (7). The disadvantages of optical systems arise from the use of cameras to track the markers. Usually, seven or more cameras are used to capture whole body motion. Each camera sees a limited space, and if the performer moves out of that space, the data are lost. Thus, the motion capture area is limited by the placement of cameras and their fields of view. Shadowing is another problem that plagues optical systems. The cameras must be able to see the markers so that the video processor and computer can calculate their positions. If the person assumes a position where a marker is obscured (shadowed) from enough cameras, the position data for that marker are lost (8, 9). Whereas in shadowing there are not enough data, optical systems can get conflicting additional data in the form of environmental light noise. Lights in the environment or reflections off of shiny or metallic objects can be interpreted as a marker (8, 9). Finally, most optical systems (except Optotrak) require the solving of the correspondence 14 problem by a computer before 3D data are available (8, 9). Each camera passively sees the markers in the scene. The video processor then calculates the 2D coordinates of the markers for each camera. Finally, the computer must take all the 2D data and find the corresponding 3D positional coordinates. This is a calculation intensive process and requires about 1 min of computation (on a workstation) for each minute of recorded whole body motion capture (10, 11). Further problems may be caused by markers placed on clothing. Clothing can move freely over the body and thus cause errors in marker drift over time. In an attempt to solve this problem, the markers are usually attached to a tight fitting Lycra bodysuit, which may be uncomfortable, and still subject to drift problems. Optical motion capture systems tend to be more expensive than others available today. Motion Analysis estimates the cost of their whole body tracking system, Expert Vision HiRes, to be around $125,000 - $137,000 (10). Adaptive Optics Associates offers the Multi-Trax system for $160,000 - $200,000 (11). Northern Digital, makers of the Optotrak, estimate that it would cost $160,000 for a tethered, whole body tracking system (12). 3 . 3 . 2 xMagnetic Whole body magnetic trackers are primarily available today from two companies: Polhemus and Ascension. These systems use perpendicular electromagnetic coils to generate defined magnetic fields in a volume. Polhemus uses AC magnetic fields, whereas Ascension uses pulsed DC magnetic fields. The performer typically wears a bodysuit with receivers attached on the limbs. The receivers contain perpendicular coils of wire which acquire induced currents as the receivers pass through the magnetic fields in the workspace. A tether or wireless link connects the receivers to the processor unit. The receiver currents are used by the processor to calculate the position and orientation 15 of each receiver. These data are then sent to a computer. A typical system is shown in Figure 3-2. Both Polhemus and Ascension systems have this general setup (13, 14). Currently, magnetic trackers are popular and many software and hardware resources are available to interface with these systems. The receivers worn on the body are fairly small (approximately 1 inch1). Each sensor requires a set of wires for data transmission, so these systems tend to be a bit more encumbering than the passive P r o c e s s o r Figure 3-2. Typical magnetic system setup. 16 optical systems. However, magnetic systems do not suffer from the shadowing problems which interfere with optical systems. Since the human body is largely permeable to magnetic fields, one cannot fully obscure a sensor measurement by blocking it with the body (8, 9). Problems w;ith these systems result from their inherent dependence on magnetic fields. The strength of the fields produced by the emitter falls off geometrically as the receiver distance increases (9). The accuracy and resolution thus depend on the receiver separation from the emitter (8, 9, 13, 14). This also limits the usable range of the receivers to within about 10 ft of the emitter (13, 14). Magnetic systems are also susceptible to magnetic and metallic interference within the work environment (8, 9). Both systems recommend keeping the emitters and receivers away from all metallic objects; electronic equipment; and floors, walls, and ceilings (13, 14). Polhemus also recommends having a professional calibration of the work area done, in order to ensure accurate results. For best results, the calibrated area should not be moved or changed in any way (15). Others have also found that the most accurate results from an Ascension system are obtained after calibration of the work area (16). The reported abilities of magnetic trackers are also the subject of much debate. Polhemus claims an accuracy of 3 inches, 3° (at 15 ft); resolution of 0.25 inch, 0.1° (at 15 ft); and a lag of 6 msec (ULTRATRACK PRO) (13). Ascension claims an accuracy of 0.6 inch, 1° (at 10 ft); resolution of 0.1 inch, 0.2° (at 10 ft) (MotionStar Wireless) (14). However, many researchers have found that these specifications tend to vary greatly depending on the environment and system setup (8, 9). A Polhemus system will cost around $36,000 for a tethered system, to $82,000 for a wireless version (15). Likewise, Ascension's Motion Star costs around $42,000 for a tethered system, to $68,000 for a wireless version (17). 17 3 . 3 . 3 Exoskeletal Instead of measuring the positions of sensors in space, exoskeletal systems measure joint angles directly by placing sensors on or around the joints of the body. The exoskeleton can be a Lycra body suit, within which the sensors are sewn, or a semirigid frame with plates, links, and sensors assembled to follow the motion of the body. The data captured at the joints are routed by wiring to a processor, which may or may not be worn on the body. A tether or wireless link connects the exoskeleton to a computer. A typical system setup is shown in Figure 3-3. Exoskeletal systems tend to be simpler than others, since the sensors directly measure the desired value, joint angle. This simplifies the electronics and algorithms required for motion capture. The range of the performer is only limited by the length of P r o c e s s o r S e n s o r s a n d J a t t a c h m e nt p l a t e s J o i n t a n g l e d a t a C o m p u t e r Figure 3-3. Typical exoskeletal system setup. 18 their tether or wireless link. Hence, their workspace is essentially unlimited. Environmental problems such as lighting, shadowing, magnetic fields, metallic objects, and others do not affect exoskeletons, since their sensors measure joint angles directly. (Note: Our sensors can be affected by very close (<0.5 inch) magnets or ferrous metals.) These systems are also fairly portable and easy to setup, since they require only the exoskeleton and a computer. Unfortunately, exoskeletons by themselves are only relative positioning devices. The joint angles are generally referenced to a point on the body, such as the center of the hips. Thus, although an exoskeleton may determine the joint angles required to walk across the room, it does not know where in the room the user has walked. The sensors for exoskeletons are necessarily mounted on or around the joints, since the sensors measure joint angles. This is more encumbering than other systems and may even limit the performer's movement to some degree. Finally, exoskeletons are difficult to design, because of the large differences of height and body proportions between individuals. To be generally useful, an exoskeleton cannot be custom made to an individual; instead, it must fit a variety of body types and sizes. Current exoskeletal motion capture devices have been largely limited to isolated, experimental devices (18, 19). Very few have been advertised or have been made available commercially. Johnson Kinetics sells a customizable bodysuit which uses electromagnetic coil pairs to sense joint angles for about $16,000 (20). TCAS, out of England, sells the Datawear for about $30,000, which is a bodysuit that uses conductive elastomer sensors to measure joint angles (18, 19). Recently, Virtual Technologies Inc., makers of the CyberGlove, has released information about the CyberSuit, a whole body motion capture suit based on their resistive bend sensor technology. It will not be available until mid-1998 and is estimated to cost around $50,000 (21). SRC estimates it could sell a production version of the SenSuit™ for around $40,000. 19 3 . 4 S e n S u i t™ SRC decided to build an exoskeletal SenSuit™ based on its needs and normal working environment. SRC needed a motion capture system to program computer graphics and entertainment robots. In addition, some SRC robots and animations are driven in real-time to interact with a live audience. Thus, most optical systems were unsuitable, since they required a significant amount of processing time (minutes) to process the motion capture data. The working environment for SRCs motion capture device is variable. Often, it requires moving from building to building or different areas of the same building. Sometimes robot programming is done at a worksite in another city. Also, the environment may not be stable in terms of lighting and magnetic fields. The motion capture area may have computers, robots, wiring, lights, and metal. Thus, SRC required a system that was easy to setup and was not affected by the environment. This excluded optical systems as well as magnetic systems. At the time, there were no other known commercially available exoskeletal motion capture systems. 3 . 4 . 1 Previous Versions SRC has built three previous versions of the SenSuit™. The first SenSuit™ only measured upper body DOFs, whereas the second version also measured the waist and some leg movements. The third version measured 32 DOFs, including the head, arms, waist, and legs. Figure 3-4 shows the second and third versions of the SRC SenSuit™. I began work on this project during the completion of the third SenSuit™. After working with and wearing this SenSuit™, it became apparent that there were still many unresolved issues. Discussions with other SenSuit™ users and engineers confirmed these problems. The problems with the third version SenSuit™ included the following: 1. Unstable soft tissue interface 20 21 2. Inaccurate joint angle measurement 3. Large and unreliable sensors 4. Poor user comfort due to weight, bulk, and design 5. Poor reliability due to plate, joint, and wire breakage 3 . 4 . 2 Goals of the C u r r e n t SenSuit™ Version My primary task was to redesign the SenSuit™ while addressing the issues described above. Figure 3-5 shows the current, fourth version of the SenSuit™. The new SenSuit™ incorporates a completely redesigned soft tissue interface. We sought to improve user comfort with the soft tissue interface by using lighter materials, substantial padding, and an ergonomic design. At the same time, we wanted plate movement minimized. To this end, we set a goal of ±5° for measurement repeatability and ±5° for measurement drift, which is primarily determined by plate movement. Small, robust joint angle sensors were created based on Hall effect technology. The sensors would have a linearity better than 3%, with noise below 0.1%. Joint angle measurement has also been redesigned for many parts of the SenSuit™, and software algorithms have been included to reduce certain joint nonlinearities. Table 3-1 lists the joints of the body, and the 34 DOFs we planned to measure using the new SenSuit™. A goal of ±10° was set for measurement accuracy. Stronger materials, protected wiring, and design modifications were used to reduce the chances of component breakage. The following chapters visit each of the above issues in detail and describe the efforts to solve the problems encountered. A B Figure 3-5. Current (fourth) version of the SRC SenSuit (A) Front. (B) Back. Table 3-1. The joints and corresponding DOFs measured in the current SenSuit™. J o i n t D O F s Head/Neck Flex/extension Ab/adduction Rotation Waist Flex/extension Ab/adduction Rotation Right/Left Shoulder Flex/extension Ab/adduction Rotation Right/Left Elbow Flexion Right/Left Wrist Flex/extension Ab/adduction Rotation Right/Left Hip Flex/extension Ab/adduction Rotation Right/Left Knee Flexion Right/Left Ankle Flex/extension Ab/adduction Rotation 4 . S O F T T I S S U E I N T E R F A C E The soft tissue interface forms the foundation of the SenSuit™. It is in contact with the user and must provide a consistent, accurate representation of the underlying skeletal movements. Accurate representation of skeletal movement will result in accurate representation of joint angles by attached sensors. However, in the effort to accurately represent the motions of the skeleton, the comfort of the user should not be compromised. 4 . 1 A p p r o a c h The goal of an exoskeleton is to follow the exact motion of the user's skeleton. This will lead to the best measurement of the angle between bones, and thus the angle of the joint. Unfortunately, short of bone screws, it is impossible to exactly match the motion of the skeleton. Soft tissues, such as muscle, fat, and skin, lie on top of the skeleton and can move independently of it. There is great variability in the amount of muscle and fat among individuals, even if they are of similar height and build. Also, muscles are active structures, changing their geometry depending on the angle of the joint. These considerations make it difficult to locate attachment points for plates or sensors. One must consider varying body types, passive motion of the soft tissue, active movement of the muscles, and user comfort. Since the SenSuit™ plates must be attached in a consistent way, and sensor errors from soft tissue motion must be minimized, we observed the behavior of the soft tissue and skeleton throughout the body. It was found that there are certain bony protrusions where the skeleton is near the surface of the skin. These skeletal landmarks are found on most people, but may be hidden in the case of overweight individuals. 25 Examples include the elbow, the anterior portion of the tibia, the back of the hand, and the sternum (see Figure 4-1). Skeletal landmarks on the surface of the body are an excellent way to provide a consistent fit between individuals. One must design an attachment plate in such a way that it lies on top of or aligns itself to a skeletal landmark. This way, there is no arbitrary placement of the plate. Muscle geometry is another important factor. For example, it would be inadvisable to attach a plate to the biceps muscle, because there is an obvious volume change depending on the flexion of the lower arm. The plate would move with the muscle as well as the underlying bone. However, the side of the upper arm does not have much muscle and remains relatively flat as the lower arm moves. Attachment to these flat muscle locations will reduce the likelihood that the SenSuit™ moves relative to the skeleton, resulting in more accurate joint angles. Previously, attaching plates to the user was an arduous process. An assistant spent several minutes adjusting sliders on the straps in order to get a tight fit on the user. Now, all plates are attached to the user with elastic Velcro straps, which allows the user to put the plate on, adjust it, or take it off in a matter of seconds. The elastic nature of the Velcro straps not only makes the straps more comfortable for the user but also accommodates changing muscle volumes during limb motions. This insures a tight fit regardless of limb movement. Bony protrusions and flat muscle locations guided the development of the soft tissue interface. This, along with appropriate plate and strap materials, created a soft tissue interface that had a consistent fit between different users, accurately tracked the skeleton despite passive and active movement of the soft tissue, was stable over time, and was comfortable enough to wear for many hours. Details about individual plates will be shown in subsequent sections of this chapter. 26 Figure 4-1. Bony protrusions used to guide the placement of SenSuit™ plates. Adapted from (1). 27 4 . 2 P l a t e M a t e r i a l The material used to build the plates was a key issue. The best aspects of previous SenSuits™, other motion capture device interfaces, and orthopedic devices were examined. One way to achieve intimate contact with the skin is through a Lycra body suit. Lycra suits are used by other motion capture devices (magnetic and optical, some exoskeletal), on top of which sensors or markers are mounted. However, Lycra suffers from stability issues. It is unpredictable how the same Lycra suit will fit multiple users and if the sensors will be positioned correctly each time. In addition, after the user moves around in the Lycra suit, the Lycra may move as it settles into place, thus changing the locations of sensors. Also, since our sensors were rigid, it was difficult to attach them to Lycra. An orthopedic device that was used in older SenSuits™ was the neoprene elbow or knee brace. This attached above and below the joint and had a hole to fit the elbow or patella. These features were advantageous in that they allowed the brace to be put on in a consistent manner from person to person (using the elbow or patella as a landmark) and enabled the brace to resist movement while it was being used (again because the elbow or patella would hold it in place). In addition, these braces were easily placed, adjusted, and removed because they had adjustable Velcro straps. Unfortunately, like Lycra, it was difficult to attach rigid sensors and links to the brace due to its soft nature. It was concluded that soft, cloth-like materials either were not stable enough or were too difficult to attach to with our rigid links and sensors. The alternative was to use rigid plates, which could be affixed to the user by straps. We assumed that properly attached plates would give the most accurate measurement of joint angles, while serving as rigid locations for the attachment of various links and sensors. The disadvantage would be the greater encumberment and discomfort of the user. 28 Previous SenSuits™ also used rigid plates. There were three varieties: aluminum filled polyurethane, ABS sheets, and aluminum. AH three of these were unacceptable for a variety of reasons. Aluminum filled polyurethane plates were molded specifically for the torso and shoulder sections of the third SenSuit™. They were made thick (0.25 - 0.75 inch) for strength, which made their weight relatively heavy (2.6 lb total). This gave the plates unwanted inertia which would cause them to shift as the user moved. The shoulder sensors would sense this shift, resulting in erroneous joint angle readings. The plate weight and rigidity also made them quite uncomfortable, because all the weight was supported on the unpadded shoulders. The third SenSuit™ also used thin (0.125 inch) ABS sheets which were used for the plates of the elbows, knees, and wrists. They were simple to make and modify by heating, bending, and cutting into the desired shape. However, they were too weak, and many snapped or were ineffective at retaining screws. For older SenSuits™, 7075 aluminum plates were used for the elbows, feet, and shoulders. Although these were strong and had no problems retaining screws, they were relatively heavy and uncomfortable. Like the thick plastic plates, their weight gave them inertia to move about independently of the body, causing erroneous sensor readings. They were sometimes painful because they did not yield during certain user movements. It was desired that the new plate material have the following characteristics: easily made in a variety of shapes, light for user comfort, controllable anisotropic properties, effective retention of screws and rivets, and sufficient strength to withstand wear and tear. The material that was chosen was a carbon fiber composite. Carbon fiber composite is routinely used in knee and arm braces after orthopedic surgery and for modern prosthetic limbs. It is made by laying up various layers of nylon, carbon fiber, and Spectra fiber and infusing them with an acrylic matrix. Once the matrix sets, it acts 29 to hold the fibers in the 3D structure in which they were molded (see Figure 4-2). Then, when loads are applied to the composite, the carbon fibers, which are very strong, take the stresses along the fiber direction. The strength of the composite is comparable to aluminum and its density is comparable to other plastics (see Table 4-1). Carbon fiber composite can be readily made in any shape and can be deformed after it has set with the application of heat and stress. It can be readily machined, drilled, and sanded and is effective at holding screws and rivets. Its properties can be made anisotropic by varying the number and direction of fibers within the matrix. Anisotropic properties are important because plates can be reinforced or weakened O u t e r Surface Acrylic Matrix Throughout I n n e r Surface F i b e r Figure 4-2. The general structure of current SenSuit™ plates. Nylon is used to provide a smooth surface finish, carbon fiber is used for structural strength, and Spectra fiber is used for better screw retention. Individual plates may vary in composition and thickness. Table 4-1. Selected material properties of SenSuit™ plates. Material D e n s i t y ( k g / m 3 ) Y o u n g ' s Modulus (GPa) U l t i m a t e S t r e n g t h (MPa) Polvurethane (22) 1030-1500 0.07-0.7 69-103 ABS (22) 1030-1060 2.1-2.8 30-52 7075 Aluminum (23) 2800 72 570 Carbon Fiber (Biaxial) 1450 60 500 30 where needed. For instance, extra fibers can be laid up where the plate joins sensor links, for increased rigidity and screw retention, but where the plate lies on the user's limb, less fibers can improve fit and comfort by allowing the plate to bend. 4 . 3 D e s i g n The following sections discuss the design of the SenSuit™ soft tissue interface in detail. Each section covers a different segment of the body and the design considerations that were important to create a consistent, stable, and comfortable soft tissue interface. Pictures in this section show the sensors attached to the plates; however, the sensors and joint angle measurement will be discussed in Chapters 5 and 6. 4 . 3 . 1 Torso The torso forms the measurement base for the upper SenSuit™ sections. The angles between the torso and hips become the waist angles, and the torso is the first link connecting to the arms and head. It is therefore essential to track as closely as possible the movements of the torso, for errors in its position affect head, shoulder, and waist measurements. Previous SenSuit™ designs had a plate on the sternum, and used straps to secure the torso plate to the user's upper body (see Figure 4-3) . There were two problems with this approach. The first problem was the size and weight of the plate. The plate was constantly pulling the torso harness forward because of its weight, and this caused the suit to shift on the user's body. Also, the inertia of the plate would further cause it to move unexpectedly if the user made a sudden movement or bent forward at too large an angle. The second problem involved the use of straps to hold the sternum plate in place. The plate had straps that went over the shoulders and around the user's sides. The problem was that when users shrugged their shoulders, they would cause the plate to move. Also, because of the weight of the plate, the shoulder straps bit into users, 31 Figure 4-3. Previous SenSuit™ torso plate. Straps over the shoulders and around the sides attempted to hold the plate in place. However, plate inertia and shoulder movement could move the plate independently of the torso. making them uncomfortable. Movement of the torso plate due to inertia or shoulder movement resulted in joint angle errors for the head, shoulders, and waist. In the design of the new torso plates, we needed to avoid resting straps or plates on the shoulders, because of user fatigue and movement caused by shoulder shrug. Also, a single plate would inevitably pull the suit forward, leading to stability problems. Therefore, two plates were used, one on the sternum and one on the spinal column between the shoulder blades. This balanced the weight from front to back. Both plates have a thin vertical section (to lie on the sternum or vertebral column) and an inverted T section at the bottom. The inverted T section anchors below the pectoralis muscles on the front (or breasts in the case of women) and anchors below the shoulder blades on the back. We made the T sections more flexible by reducing the amount of carbon fiber 32 in these sections. This allows a more comfortable and customizable fit around the user's chest. The front and back plates are connected via two stiff links that travel over the shoulders. Extra unidirectional carbon fiber was layed in the links to stiffen them in the direction of bending. The plates and links are molded as one piece for strength (see Figure 4-4). The links are flexible enough to allow the user to spread the plates apart to fit over their head, but stiff enough to have the plates mildly squeeze the user from the front and back. The torso plates are designed so that when properly worn, the links lie about 0.5-1.0 inch above the shoulders. When shrugging, the shoulders come close to the links but do not touch. The fit of the plates is enhanced by the use of two sets of straps, both of which travel along the ribs, under the wearer's arms (see Figure 4-4). The use of light carbon fiber and clamping on the chest distributes the weight of A B Figure 4-4. Current torso plates. (A) The torso plates alone. They are made from a single lay-up of carbon fiber. (B) Proper placement of the torso plates leaves a gap between the shoulders and the links. 33 the torso plates, links, and sensors over a wide area, making the fit much more comfortable for the user compared to the previous SenSuit™. Since the previous SenSuit™ rested the torso plates on the shoulders, users complained of shoulder pain after only 2 to 3 h of use. Users have worn the new design for 8 h with no complaints. The new torso plates incorporate a variety of improvements: dual carbon fiber plates, squeezing the user with the hinge and straps rather than hanging the torso section, a single lay-up design, and avoiding the pectoralis muscles, shoulders, and shoulder blades through conscientious design. These improvements allow the torso plates to faithfully follow the motion of the upper body without interference caused by excessive inertia, plate drift, or shoulder movements. 4 . 3 . 2 Head The head presents very minor attachment problems compared with other parts of the body. The skull is covered by a thin layer of skin, with very little muscle or fat tissue. The simplest attachment to the skull is with the use of a baseball hat, which was used in previous SenSuits™ (see Figure 4-5) . The sensors and links were attached to the top of the hat. Unfortunately, since the hat had a cloth top, it allowed independent movement of the attached sensor. Movement was reduced by making the hat very tight fitting, but this was uncomfortable over extended use. It was decided that the sensor must be attached to a stiff head piece in order to reliably track skull movements. Fortunately, a stiff head piece is readily available in the form of a head strap that is used to secure laboratory face shields. The strap has an adjustable diameter with a plastic piece that crosses over the top of the head. For use with the SenSuit™, the face shield was removed, and the head axial rotation sensor was mounted to the top of the cross piece (see Figure 4-6) . The resulting head piece was lined with 0.125 inch neoprene padding to reduce Figure 4-5. Baseball style hat used in previous SenSuits™. Figure 4-6. Current head attachment. The head rotation sensor mounts on the cross piece. The diameter is easily changed with the adjustment knob. any stress concentrations between the plastic and the user's head. The stiffness inherent in the plastic crosspiece reduces independent movement of the sensors. The shoulder was the most difficult joint for which to design the soft tissue interface. The problem stems from the many DOFs found in the shoulder, along with the large ROMs of those DOFs (see Figure 4-7). Since we were interested in measuring the three shoulder movements (flex/extension, ab/adduction, and rotation) rather than the shoulder girdle movements (roll and shrug), the shoulder had to be isolated from the shoulder girdle (see Table 3-1 for a list of the DOFs we planned to measure). Otherwise, shoulder girdle movements might interfere with the measurement of the shoulder angles. Previous SenSuits™ attempted to isolate the shoulder by using a shoulder plate 4 . 3 . 3 Shoulder Shoulder flex/extension (+180° to-50°) J 1 Shoulder girdle rotation (roll) (+34° to-11°) Shoulder girdle ab/adduction (shrug) (+11° to -34°) Figure 4-7. Shoulder DOFs and ROMs. Adapted from (2). 36 that wrapped over the top of the shoulder, from front to back (see Figure 4 - 8 ) . This arrangement was also tried on the new SenSuit™ with carbon fiber plates. This shoulder plate arrangement had two shortcomings: shoulder geometry and muscle movement in the area. There was no single shoulder plate design that would fit the large variety of shoulder thicknesses and muscle build. Thus, there was no consistent way to place the shoulder plate, and when placed, it rarely fit the individual very well. Also, movement of the deltoid muscle, trapezius muscle, and shoulder blade constantly shifted the shoulder plate. Therefore, the shoulder attachment needed to be redesigned to fit each user consistently and reduce the effects of muscle movements around the shoulder. Figure 4-8. Failed shoulder soft tissue interface (third SenSuit™). 37 There are two bones that can act as skeletal landmarks for the shoulder girdle, the acromion and the clavicle. These bones are covered only by a layer of skin, and the nearby muscles do not interfere with them when flexed. The clavicle, however, is too far forward to rest a plate on and, when shrugging, left little room between itself and the neck. These difficulties in attaching to the clavicle led us to use the acromion. The shoulder plate is actually the end of a 0.25-inch diameter aluminum rod with 0.25 inch of padding wrapped around it (see Figure 4-9). This gives it an area of about 0.5 x 1.0 inch to rest on the acromion. There are two elastic Velcro straps attached to the rod, which are also attached to the front and back torso plates. They provide a constant compressive force between the rod and the acromion, ensuring that the plate follows the shoulder girdle movements. The movement of the rod is further restricted Figure 4-9. Current shoulder attachment. The new design rests on the acromion. Its motion is restricted by the rod that connects it to the front torso plate, which mimics the motions of the sterno-clavicular joint. Two elastic straps provide a downward force to keep it firmly attached to the shoulder. The wiring cable for the sensor cluster has been removed for clarity. 38 by the universal joint which attaches it to the front torso plate. The rod mimics the motions of the sterno-clavicular joint, allowing only up/down and front/back motions (see Figure 4-9). The universal joint does not allow rotation or sliding. The rod is also bent so that it touches the shoulder only on the acromion. Arm movements and muscle flexions around the shoulder do not interfere with the rod's ability to track the shoulder girdle. The combination of mimicking the sterno-clavicular joint, staying free of the arm and associated muscles, resting on the acromion, and holding the plate in place with straps allows the new shoulder plate to faithfully track the motions of the shoulder girdle on all users, without interference from muscle movements. This allows the sensor cluster, which is mounted on the rod at its most lateral and superior aspect, to track the shoulder movements without measuring shoulder girdle interference. 4 . 3 . 4 Elbow The elbow plates track the motions of the humerus and the radius/ulna. Their motion is used to find the elbow flexion angle as well as the angles of the shoulder DOFs. Previously, the SenSuit™ used only a single plate attached in an arbitrary location on the forearm. This plate was connected on its proximal end to the shoulder plate via the elbow rotation sensor, the three sensors in the shoulder, and a series of linkages (see Figure 4-10). Its distal end was connected to the wrist plate by a stiff cable. The elbow plate had a curved cross section and no identifying features with which to orient it with the arm. Thus, placement of the plate was inconsistent. Also, the plate had a tendency to wander as the user moved, since it was not properly constrained. Two plates were used for the redesigned elbow interface, one above and one below the elbow. This helps to constrain the movements of the plates to rotation only 39 Figure 4-10. Previous elbow plate and associated linkages. about the elbow axis. The muscles around the elbow (biceps, triceps, brachialis, etc.) almost completely cover the humerus, radius, and ulna; therefore, the plates had to be located where muscle movement would be minimized. After observing the motions of the muscles around the elbow, it was decided that the best placement of the plates would be on the lateral aspect of the upper arm and the dorsal aspect of the lower arm. These two locations had the least amount of movement during elbow flex/extension and wrist rotation. The point of the elbow, or olecranon, was used as a reference to align the lower plate (see Figure 4-11). The two plate design has two benefits. First, the user can now feel when the elbow sensor is aligned with the elbow joint, because a misalignment causes the plates to pull on the skin. The user adjusts the location of the elbow assembly such that they feel no plate movement as the elbow bends. Second, plate drift has been reduced. The dual plate design stabilizes the assembly on the joint, and the plates no longer visibly 40 Figure 4-11. Current elbow attachment. The two plates are perpendicular to each other, one lying on the lateral aspect of the upper arm, the other on the dorsal aspect of the lower arm. The lower plate has a cut to align it with the olecranon. move over time (compared with previous SenSuits™, where plates obviously moved during motions, and constantly needed to be readjusted). This is due to the additional constraints created from using two plates. The plates are connected by the sensor joint, which only allows for rotation about its axis. Off axis movement of one plate also moves the other plate, which means that there is twice the resistance from the two sets of straps. Also, since the plates are perpendicular to each other, they constrain movement in different ways. The top plate restricts medial and lateral movement, being mounted 41 on the side of the upper arm. The bottom plate resists anterior and posterior movement, being mounted behind the lower arm. 4 . 3 . 5 W r i s t / H a nd The wrist has three movements: flex/extension, ab/adduction, and rotation. To track flex/extension and ab/adduction, the motion between the hand and wrist needed to be measured. To measure wrist rotation, the forearm rotation relative to the elbow had to be measured. The previous SenSuit™ had well-designed wrist and hand plates that took advantage of skeletal landmarks. It used one plate proximal to the wrist joint that wrapped from the ulnar to the dorsal aspect and a second plate that covered the dorsal aspect of the metacarpals (back of the hand). This is the same design used by the new SenSuit™ (see Figure 4-12). Figure 4-12. Current wrist attachment. 42 The back of the hand is a logical plate attachment area, because there is only skin and tendon on top of the hand bones and the area remains relatively flat during all movements of the fingers and wrist. The wrist plate was placed around the ulnar and dorsal aspects of the forearm. The "L" shape of the plate constrained its position in the transverse directions of the lower arm. A cut in the wrist plate allowed it to be aligned with the ulnar head. The protrusion of the ulnar head also helped to prevent the wrist plate from moving longitudinally along the lower arm. 4 . 3 . 6 Hips The hips form the base for the lower SenSuit™ sections, and a firm attachment to the pelvis was required for accurate measurement of waist and leg angles. The hips are also used as the base of the kinematic chains for our computer model, so any extraneous movement of the hip attachment will cause gross errors in the model. Therefore, it is essential that the hip attachment accurately tracks the movement of the pelvis. The previous SenSuit™ used a wide, adjustable belt as the hip interface. The belt had two leather patches, onto which the hip sensors were mounted (see Figure 4- 13). The belt could be worn at various heights, and since it was hanging on the body, it could move around on the hips. The leather patches were not very stiff and were easily distorted as the legs moved. Thus the hip attachment did not have a consistent placement, was prone to drift, and could move as the legs moved. These problems lead to inaccurate measurements of waist and hip angles. To solve these problems, an attachment was needed that could be placed on the hips in a consistent manner, was rigid, and resisted drift and movement as the person moved. Figure 4-14 shows the current hip attachment. Two "C" shaped plates hold onto the side of each hip. They are connected in the back by a rigid link of carbon fiber. The 43 Figure 4-13. Previous hip attachment. link length can be adjusted by two knobs on the hip plates, to allow for variations in hip girth. A 2-inch wide belt runs around the plates. It was clasped at the front, with an adjustable buckle. A strap comes off the back of both plates, around the crotch, and snaps buckles at the front of the plates. These strap lengths are also adjustable. The hip plates are located so that the top of the plates are in line with the superior iliac crest. This ensures that the plates will be tight against the pelvis. If the plates are too high, they are supported by the waist, which is very soft. If the plates are too low, the movement of the legs and the gluteal muscles will cause the plates to move. In either case, the user will feel the instability of the hip plates and can readjust them. 44 Hip plate Belt Figure 4-14. Current hip attachment. Movement of the hip plates relative to the pelvis is reduced in a variety of ways. When the adjustment knobs are tightened, the hip plates and link form a rigid structure. Thus, if muscle or leg movement attempts to move one plate, it is resisted by the entire structure. The plates are tightened onto the hips with the belt, which runs along the outside of the plates. This uniformly pulls the plates firmly against the pelvis (see Figure 4-15) . Also, the assembly does not hang on the person, as in the third SenSuit™. Instead, the two straps that connect the back of the plates to the front prevent vertical movement. The widening of the hips and the downward pull of the straps serve to lock the hip assembly onto the pelvis (see Figure 4-15). 4 . 3 . 7 Knee The knee attachment tracks the movements of the femur and the tibia/fibula. Since links from the hips and ankles attach to the knee plates, errors of positioning or drift of the plates will affect the hip, ankle, and knee measurements. 6 Figure 4-15. Mechanisms to hold the hip assembly in place. (A) Circumferential. The plates and link form a rigid structure. The plates are pressed onto the hips when the belt is tightened. (B) Vertical. The widening of the hips pushes the plates up, while the two straps pull the plates down, locking them into position along the superior iliac crest. 46 The previous SenSuit™ had a single knee plate attached onto the proximal, anterior portion of the tibia (see Figure 4-16). As with the elbow, a single plate was not enough to prevent the attachment from moving. Gravity continually acted to pull the knee plate down, and the straps had to be uncomfortably tight in an effort to prevent drift. Also, although the plate was shaped around the tibia, it could still drift circumferentially around the lower leg. The redesigned knee attachment uses two plates, similar to the elbow. The distal plate wraps around the proximal anterior portion of the tibia and is aligned with the most proximal end of the tibia. The tibia protrudes along the length of the lower leg and thus provides a good skeletal landmark for placing the plate. The proximal plate is not attached or aligned to any bone. The femur of the upper leg is entirely covered by muscle, providing no skeletal landmarks. It was observed, however, that the large muscles of the leg, the hamstrings and the quadriceps, mainly protrude dorsally and ventrally. The least amount of variation and muscle motion occurred on the lateral Knee plates Figure 4-16. Previous SenSuit™ knee attachment. 47 aspect of the upper leg. The upper knee plate was placed at the distal lateral aspect of the upper leg (see Figure 4-17). This design was analogous to the elbow7 attachment. Similar results to the elbow design were observed, namely, drift and extraneous movement were no longer visibly a problem (compared with previous SenSuits™). Again, by using two plates, the user could feel when the knee sensor was lined up with the knee joint and easily correct the problem. 4 . 3 . 8 Foot Finally, there needed to be an attachment at the foot in order to measure ankle and foot movements. The foot plate was linked through ankle sensors to the lower knee plate. Figure 4-17. Current knee attachment. 48 The previous foot attachment was an aluminum stirrup that wrapped around the bottom and sides of the foot (see Figure 4-18). The ankle sensors were attached to the lateral side of the stirrup. The stirrup was made oversized, so that a variety of foot sizes could fit into it. Unfortunately, this made a tight fit with the foot possible only if the foot fit exactly in the stirrup. Otherwise, free space and excessive weight allowed it to move and drift on the foot. Also, the rigid metal frame of the stirrup was quite uncomfortable for walking around. Additionallv, the lateral mounting of the sensors exposed them to objects in the environment, and considerable time was spent repairing accidental damage to these sensors. The redesigned foot attachment eliminates the stirrup and lateral sensor mounting. Instead, the plate is attached to the top of the foot (see Figure 4-19). This design is far more comfortable for the user, since the foot is no longer bound by an unyielding metal stirrup. The sensors are now protected, because they reside above the foot, and cannot be damaged from the side. The plate does have a tendency to drift, since its motion is constrained only by two straps. However, because it is tightly strapped to the foot (which is not uncomfortable because shoes are worn), it does not make gross movements due to inertia or leg motion. Aluminum stirmps Figure 4-18. Previous foot attachment. 49 Foot plate Figure 4-19. Current foot attachment. 4 . 4 S u m m a r y The redesigned SenSuit™ soft tissue interface uses rigid carbon fiber plates to follow the user's body for measurement by joint angle sensors. The new plates are light (similar to plastics), which keeps the SenSuit™ weight down for user comfort and reduced inertia. In conjunction, carbon fiber is strong (similar to aluminum), which allows the plates to withstand general abuse. Another attractive property of carbon fiber was its anisotropic properties. By varying the thickness and direction of fibers, the plates could be rigid for sensor attachment or structural support, but elastic where the SenSuit™ needed to bend around the user's body. 50 The new soft tissue interface makes use of skeletal landmarks found on the body. Plates are attached onto or aligned to bones that come near the surface of the skin. Where the bone is completely covered by muscle, the plate is attached where muscle motion will have the least effect on joint angle. When possible, a plate is used above and below the joint, which adds additional constraints to the attachment, making it more stable. The use of carbon fiber plates in conjunction with these design guidelines has allowed us to create a SenSuit™ soft tissue interface that provides an unambiguous, consistent fit on the user's body. The interface resists quick, nonskeletal motions as well as long-term drift and is comfortable over many hours of use (known from interviews with users). 5 . S E N S O R S Once the design of the soft tissue interface was settled upon, I next turned to the design of the joint angle sensors. It was decided that the potentiometers used to measure angles on previous SenSuits™ would be unsuitable for the new version. The potentiometers were bulky, prone to breakage, and could become electrically noisy due to contamination and wear. This section details the design of the new joint angle sensors. 5 . 1 M o t i v a t i o n In an effort to decrease the profile and bulk of the SenSuit™, it was necessary to create small sensors to measure the joint angles. The joint sensors used on previous SenSuits™ were high linearity potentiometers, JDK model 6009-1003. Although these sensors had a good linearity (0.5%), they did have substantial bulk. Their thickness was 0.665 inch and they had a diameter of 0.875 inch. Athough this may seem small, it tended to make the potentiometers protrude from the various joints. Also, the potentiometer had to be mounted on top of the structural joint to relieve the sensor from off-axis loads which might damage its components. Together, the joint/potentiometer combination had a thickness of 1.1 inches. Additional problems encountered with the potentiometers included wiring and noise. The potentiometer wires projected from the back of the housing and would often become caught on an object in the environment and break. Some potentiometers became electrically noisy over time. This is because they are mechanical devices which use a wiper on a conductive plastic substrate to obtain linear voltages. Off axis loading, wear, 52 and liquid or particulate intrusion could cause damage to the substrate, rendering the potentiometer noisy. There were several objectives in the design of the new sensors. First, the sensors had to be reasonably linear. A linearity of less than 3% would be acceptable, because it had been estimated previously that the suit fit could cause larger errors (~5- 10%) (25). The new sensor should have a low profile (<0.25 inch) and, if possible, include the mechanical joint to save space. Although small, the sensor must be able to withstand the rigors of everyday use on most joints on the body. Also, it was desirable to have the sensor noise free (<0.1%), regardless of wear and abuse. The wiring needed to be concealed or routed in such a way that it would be difficult to snag and break. The ROM of the sensor needed to be at least 180°, to accommodate most of the joints in the body. Several types of sensor concepts were studied. Existing optical encoders were deemed too large, and creating a custom optical sensor was prohibitively expensive. Although potentiometers were ruled out, we considered the possibility of repackaging the inner components in a more compact way. However, previous researchers at SRC had found this to be exceedingly difficult (26). Regardless, they would still suffer from many of the same problems as potentiometers. A new type of electrostatic sensor (Rotary Displacement Transducer) made by SRC was also considered; however, the sensor required further development and would not be ready for integration within the SenSuit™. SRC has also utilized Hall effect sensors in rotation sensing (27). After extensive modifications, these types of sensors became the new rotational sensors within the SenSuit™. Because the sensor was integrated within a mechanical joint, it was called the HallJoint. The HallJoints satisfied all of the performance criteria defined above. The following sections explain the Hall effect theory, the general design of the HallJoints, and their performance. 53 5 . 2 T h e o r y The Hall effect was discovered in 1879 by Edwin H. Hall. It was known at the time that a magnetic field could deflect a perpendicular electron beam. Hall showed that this deflection could also take place in wires carrying current (a type of electron "beanf' in a solid) (28) (See Figure 5-1). The Hall potential difference is measured by placing a voltmeter across the strip of metal. The voltage (V) is given by: where E is the electric field, d is the strip width, B is the magnetic field, i is the current, n is the charge carrier density, e is the electric charge of an electron, and t is the strip thickness. Therefore, with constant current, the voltage is directly proportional to the perpendicular component of the magnetic field (28). The linear Hall effect sensor can be purchased as a small integrated circuit, with an internal amplifier and temperature compensation circuitry. Table 5-1 highlights the characteristics of the chip used in the SenSuit™, the SS496A1 by Honeywell Micro Switch. We choose this chip because it had the greatest sensitivity and best chip to chip consistency. The linear Hall effect chip is a small package that provides a convenient way to measure magnetic fields. Our focus then was to create a sensor that had a linear variation of the magnetic field around an arc, which we could measure with the linear Hall effect chip. This would give us our angular sensor. or V = net Bi (1), (2) 54 X B X X X A X X X X X X X X X X B X X X j X X If X X X X X Figure 5-1. The Hall effect. (A) A beam of electrons (e) moving up is deflected to the right by a magnetic field (B) pointing into the page. (B) A current (i) moves down through a strip of metal (d. width) subject to the magnetic field. Since electrons are actually drifting up (with drift speed, vd), they feel a deflecting force (FB ) to the right. (C) At equilibrium, electrons (-) have ''piled up" on the right side of the strip, causing an electric field (E) to form from left to right. The electrical force (FE ) on the electrons is balanced by the magnetic force (FB). The voltage (V) is measured across the strip. 55 Table 5-1. Linear Hall effect sensor selected characteristics. S p e c i f i c a t i o n Honeywell Micro Switch SS495A1 (29) Supply Voltage (V) 4.5 to 10.5 Supply Current (mA) 7.0 Nominal Voltage (V) 50% Supply Voltage Magnetic Range (Gauss) -670 to 670 Sensitivity (mV/Gauss) 3.125 ±0.094 ( a t 5V supply) Linearity (%) 1.0 Frequency Response (Hz) > 10k Length, Width, Height (inches) 0.118 x 0.160 x 0.620 5 . 3 G e n e r a l D e s i g n Figure 5-2 depicts previous SRC versions of rotary displacement transducers using Hall effect sensors as the sensing elements. Both designs utilize magnetically strong samarium-cobalt magnets to generate the magnetic field sensed by the Hall effect sensor. In one case, ferrous "rails" guide the magnetic flux around a loop, while the other had exposed magnets. Both of these designs required mounting onto a larger A Top V i e w B Figure 5-2. Prior SRC rotary displacement transducers using Hall effect sensors. (A) This design utilizes rails to increase the ROM. (B) This design eliminates the rails. but has a smaller ROM. 56 mechanical joint, and both designs had a limited ROM. Since it was desired to have an integrated sensor and joint and a ROM of at least 180°, the previous designs were unsuitable for the SenSuit™. We decided to use the rail idea and combine it into the mechanical joint. The rails gave us increased ROM and better linearity than the no-rail design. We experimented with a variety of geometries before settling on the HallJoint illustrated in Figure 5-3. This new angular sensor satisfied the physical and electrical performance criteria we established earlier. The rails and magnet side link are screwed together to form a rigid structure, and Figure 5-3. The HallJoint. (I) Complete HallJoint. (II) Exploded view. The joint consists of two aluminum links, the magnet side link (A) and the sensor side link (B). The steel shaft (C) is press fit into the sensor link and the Hall effect sensor (D) is fixed into place in the adjacent slot. Wires (not shown) are soldered onto the sensor leads, and run along the link. The two steel flux rails (E) fit on the shaft with a slip fit, which allows for smooth rotation of the rails. The magnet side link holds two oppositely polarized 18 MGO samarium-cobalt magnets (F). The link and magnets are sandwiched between the ends of the flux rails, which are held together by two small screws (G). Mounting holes (H) allow the links to be attached to other objects. 57 rotate about the shaft in the sensor side link. The rails serve to carry the magnetic flux of the magnets around the loop and through the Hall effect sensor. Because the magnets have opposite polarizations, the flux around the loop varies in strength and sign, canceling out to zero in the nominal position. The Hall effect sensor, embedded in the sensor side link and under the loop of the rails, cuts through the magnetic flux. The flux changes linearly around the loop, which leads to a linearly changing output voltage from the Hall effect sensor. This is interpreted by the computer as a changing angular output (See Figure 5-4) . Next, we will discuss how well the new HallJoint meets our performance criteria. Flux Rail s Magnet N 1 1 • • • 1 1 Flux Rail Side view - flux rails "unwrapped' N agi S Magnet B o CO o CO c O CO + 1 6 0 - 1 6 0 A n g l e ( d e g r e e s ) 1 8 0 Figure 5-4. Interpreting angular changes from a HallJoint. (A) View of the magnet flux between the rails (with the rail loops spread apart). The linear Hall effect sensor travels between the rails, cutting through the varying magnetic field. (B) Graph of magnetic field intensity as a function of angle. 58 5 . 4 P e r f o r m a n c e The design criteria for the new sensors included linearity better than 3%, noise < 0.1%, resistance to wear and contaminants, thin (<0.25 inch), strength to withstand to the rigors of everyday use, concealed wiring, and ROM >= 180°. 5 . 4 . 1 Electrical The HallJoints were tested for linearity on a potentiometer testing machine at SRC. The HallJoint was coupled to a 432,000 divisions/revolution optical encoder, which served as an absolute angular reference. An electric motor moved both devices in unison. The HallJoint analog output was buffered by a unity gain amplifier and then digitized by a 12-bit A/D card. A computer recorded the digital output from both devices. The linearity test consisted of moving the HallJoint through its range of motion and comparing its output to that of the encoder. Figure 5-5 shows a typical result of this test. The linearity is defined as the maximal difference between the sensor output and the best fit line, divided by the sensor range. In this case, the maximum difference was 0.038 V, over the sensor range of 1.93 V, giving a linearity of 2.0%. The linearity of the first HallJoint based on the above design was about 4%. To improve the linearity, we modified the shapes of the rails. The outer edge of the loop on the rails was moved closer to the straight sections. After limited trial and error, we succeeded in lowering the linearity to the current 2.0%. It is expected that further modifications to the geometry of the flux rails might improve the HallJoint linearity to that of the Hall effect sensor, or 1.0%. It is interesting to note that unlike potentiometers, the Hall effect sensors will never get noisy due to mechanical wear of the sensing element. This is because the Hall effect sensor actually floats in the sensor side link, sensing magnetic flux while never 59 4 -I ' ' 1 1 1 1 ' ' } 1 ' 1 ' ' 1 1 • 1 0 30 60 90 120 150 180 Angle (deg) Figure 5-5. HallJoint linearitv. actually touching the rails. Therefore, sensor life is expected to be limited by a physical breakdown of the HallJoint, not a failure of the sensing element. Tests were performed to roughly determine the inherent noise of the HallJoint and sources of noise in the environment that might affect the HallJoint. To determine the inherent noise of a HallJoint, the SenSuit™ was hooked up to a data acquisition program. The data acquisition program was used to record the raw, digitized output from a sample HallJoint. First, two 10KH resistors were used as a voltage divider and bypassed the sample HallJoint. The voltage divider served as a control to distinguish between noise inherent to the electronics and noise added by the HallJoint. Five hundred and sixty samples were taken with the voltage divider. Then, the sample HallJoint was replaced and positioned near the midpoint of the sensor. Three trials of about 600 data points were performed with the sample HallJoint. Table 5-2 shows the statistics of the trials. Note that the average differs from trial to trial, since the HallJoint 60 Table 5-2. Statistics for the HallJoint noise test. C o n t r o l Trial 1 Trial 2 Trial 3 Average (counts) 2003 1911 1872 2069 St Dev (counts) 0.42 0.65 0.61 0.62 Range (counts) 2 4 4 3 S a m p l e s 560 593 644 620 was moved between trials. The average of the HallJoint trial standard deviations was 0.63 counts. Compared to the voltage divider control of 0.43 counts, it appeared that the HallJoint added some noise to the electronics. To find the noise contribution of the HallJoint, we assumed that the noise from the electronics and the HallJoint were independent. Then, Variance( E l e c t t o n i c s ) + V a r i a n c e(Haiij0,nu = Variance( E l e c t r o m c s + H a i U o i n t ) (3) Variance is equal to the standard deviation squared. By substituting for standard deviation and rearranging, we found: (4) Using equation 4 and the above values for the noise of the electronics and electronics plus HallJoint, we find that the standard deviation of the error due to the HallJoint is 0.46 counts. The HallJoint uses only 20% (1 Volt over an excitation voltage of 5 Volts) of the 12-bit A/D range, or about 819 counts. Thus, the noise from the HallJoint represents an error of about 0.06% over the range of the HallJoint. This satisfies our 0 . 1% goal. 61 Since HallJoints are magnetic sensors, there was concern about magnetic fields in the environment affecting the joint angle data. The magnetic flux on the Hall effect sensor within the Hall Joint reaches a maximum at about ±160 Gauss. The strengths of some common magnetic fields are shown in Table 5-3. To determine what external magnetic fields could effect the HallJoint, an experiment was performed to find the strength of an external magnetic field that would cause a 1% change in the output of the HallJoint. First, a calibration was performed to determine the magnetic field strength along the axis of a small magnet of the same type and size as used in the HallJoints. A linear Hall effect sensor measured the magnetic field at various distances from the magnet. By graphing the results and fitting them with a power function, we found an equation that describes the magnetic field strength along the axis of the magnet (see Figure 5-6). It should be noted that the calibration is not valid for distances closer than 0.1 inch. The Hall effect sensor saturates at field strengths above about 670 Gauss, which occurred at distances closer than 0.1 inch. The external magnetic fields interacting with the HallJoint could be controlled by varying the distance of the calibrated magnet. The axis of the magnet was aligned over the Hall effect sensor within the joint. At about 1 inch from the flux rail, the output of the HallJoint changed by 0.7% (% of the full ROM). The magnetic field at 1 inch from the magnet was about 4.5 Gauss. This was far greater than any of the common magnetic fields noted in Table 5-3. Ferrous metal at any distance did not have a measurable effect on the HallJoint. Table 5-3. Strengths of common magnetic fields. Object D i s t a n c e Field Strength Earth (28) Sea Level 1 Gauss Computer Monitor (30) 1 foot 2-6 milliGauss Fluorescent Lights (30) 1 foot 6-30 milliGauss 62 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 D i s t a n c e from Magnet (inches) Figure 5-6. Strength of the magnetic field along the axis of the test magnet. However, when ferrous metal touched a flux rail on the HallJoint, the effects were significant. The output of the HallJoint could vary by as much as 27% (% of the full ROM) when touching ferrous metal. A change of 60% was possible when a magnet (same magnet as above) touched the flux rail. These results show that the HallJoints are susceptible to ferrous metals and magnets they may physically contact, so some care must be taken to avoid this complication. 5 . 4 . 2 Physical The physical dimensions of a normal HallJoint are 0.5 inch wide by 0.200 inch thick. This is a substantial decrease in bulk and profile over the JDK 6009-1003 potentiometers we used previously (see Figure 5-7). In the standard configuration, the HallJoint has a range of motion of ±90°, for a total motion of 180°. However, it is possible through the design of the links to offset the range of motion (see Figure 5-8) . 63 Potentiometer diameter • 0 ^ L A I - I - I 1 inch Potentiometer width A B C Figure 5-7. Comparison of sensor sizes. Top: Front view. Bottom: Side view. (A) Potentiometer/joint used in third SenSuit™. (B) Reinforced HallJoint used in knee (see Figure 5-11) (C) Normal HallJoint. 64 O +180' B Figure 5-8. ROM variations of the HallJoint. (A) Nominal range of motion. (B) Modifying the magnet side link offsets the range of motion by 90°. Some joints can sense axial rotation by utilizing an extended shaft (see Figure 5- 9). In this form, the sensor side link serves only to hold the Hall effect sensor and is thus as small as possible. Since the shaft is press fit into the sensor side link, turning the shaft generates a change in angle. One of the most useful forms of the HallJoint is that used for the measurement of three DOF joints, such as the shoulders and hips. It consists of three HallJoints that are integrally connected together to form a three DOF joint (see Figure 5-10) . This particular joint can then simultaneously measure three DOFs, such as flexion, abduction, and rotation of the shoulder. Figure 5-9. An extended shaft HallJoint for measuring axial rotation. The sensor side link has been shortened, and does not connect to anything. Instead, the extended shaft is rotated, allowing axial angular measurements. Although there have been no formal wear tests done on the HallJoints, some have been in use for a year without failing. For the year, the SenSuit™ has seen weekly usage and, at one point, was used daily for a month to program a series of robots. We estimate that many DOFs have seen 100,000 to 300,000 movement cycles. The HallJoints rely on a slip fit between the shaft and the flux rails for their bearing surfaces. Although the surfaces are oiled, the rail steel is still sliding on the shaft steel (the steels have different crystalline structures to prevent binding). It has been noticed that on some of the older joints, a buildup of fine metal filings and increased play are evidence of wear. The HallJoints are aligned with the joints of the body in such a w;ay as to avoid off-axis loading. Off-axis loading such as twisting and bending of the HallJoint will be transmitted through the bearing surfaces, and lead to increased wear. This has been noticed in the elbow HallJoints as increased play compared to same age HallJoints in the wrist. If the off-axis loading is extreme (> 47 in-lbs), as we found in the knees, the screws holding the joint together are loosened, the bearing surfaces become grooved, and the joint becomes more difficult to turn and can even seize motion completely. This 66 Figure 5-10. The three DOF HallJoint. It is a spherical joint, containing three successive perpendicular DOFs. All three rotation axes intersect. provoked the design of a reinforced joint, which was already shown in Figure 5-7 and is further explained below. The SenSuit™ knee joint attaches to the user at the anterior portion of the tibia and the lateral aspect of the upper leg. Since the tibia is used as a skeletal landmark, the lower plate is shaped around it as a firm attachment point. The upper leg, however, has no protruding bones, and the upper plate attaches to the soft tissue on the side of the upper leg. A problem arises from the large variability in muscle and fat content of the upper leg between individuals. Since the upper plate has only a single geometry, it may not fit each user ideally. A user with thicker or thinner upper legs than the plate was 67 designed for would induce off axis side loads on the HallJoint at the knee. Because the legs contain the strongest muscles in the body, the user would not notice the damage occurring to the knee joint. The result of the off-axis loads was to loosen the screws holding the flux rails together, as well as causing deep grooves to form on the shaft. The loosened screws made the joint wobbly, and the grooves caused it to be difficult to turn. The solution to this problem was to reinforce the HallJoint with a reinforcing clamp, which held ball bearings to take any off axis loads (see Figure 5-11). Figure 5-11. The reinforced HallJoint. The standard HallJoint (A) has a lengthened shaft (B) to be inserted into the bearings (C). The bearings reside in a reinforcing clamp (D) that fits around the HallJoint. Screws (E) hold the clamp together. 68 With this arrangement, there has been no further damage to the Hall Joints in the knees. Unfortunately, reinforcement more than doubles the thickness of the joint, to over 0.45 inch. Also, since the bearings are made from ferrous metal, they interfere with the magnetic flux of the flux rails, thereby decreasing the linearity of the HallJoint to about 3%. To determine the static strength of the HallJoints, we subjected one to a bend test. The HallJoint was mounted in an MTS Sintech 1/S uniaxial load frame, clamped at one end and subjected to forces at the other end (see Figure 5-12). This was designed to mimic the loading in the knee attachment, where one end of the HallJoint was firmly attached to the lower leg plate, and the other was bent outward by the upper leg plate. The joint remained elastic up to loads of 30 lbs, and would return to its original shape after release of the load. At 33 lbs. (end moment of 47 in-lbs), the HallJoint yielded. It failed at both ends of the aluminum links, and the screws holding the flux rails together became loosened (see Figure 5-12). The joint did not break apart; but it was unusable after it yielded because the damaged links interfered with normal movement. The static test did not create any noticeable damage to the shaft. The grooving Figure 5-12. HallJoint static strength test and failure mode. The HallJoint was screwed onto a rigid frame at the left. A wedge exerted forces on the inside mounting hole at the other end. Arrows show the various failures. Screws loosened Link bent Link bent 69 observed in the knee HallJoint probably occurred because of off-axis loading in combination with forced rotation. The rails may have spread apart slightly, bringing the rail edge into contact with the shaft. Upon rotation, the edge of the rail created grooves on the shaft, resulting in a high friction bearing surface. Again, this problem has only been observed in the knee attachments and has been solved by using the reinforcing clamp. 5 . 5 S u m m a r y The HallJoint is well suited for its role in the SenSuit™. It is a compact, versatile joint angle sensor and is used on 33 joints throughout the SenSuit™. Electrically, the HallJoint uses a small Hall effect sensor to output a linear voltage depending on the joint angle. The HallJoint has a linearity of 2% and adds little noise to the electrical system (0.06%). The HallJoint is unaffected by normal magnetic fields within the working environment, such as those emitted by computer monitors, fluorescent lights, or even the earth. However, magnets or ferrous metals physically touching the sensors could have an adverse effect on its output. The Hall effect sensor itself is sealed within a link of the HallJoint, thus mechanical wear and contaminants cannot affect the sensing element. Mechanically, the HallJoint is an integrated sensor and joint, which saves space and weight due to its small size. By varying the geometry of the links attached to the flux rails, the ROM may be offset, or specialized sensors can be constructed, as with the three DOF sensor cluster. The slip fit that the sensors depend on for smooth rotation is vulnerable to off axis loads; however, the SenSuit™ is designed so that the sensors do not normally experience off axis loading. Unfortunately, the large variation in knee joint geometry caused the knee sensor to fail. A bearing reinforced joint was designed which can handle the increased off axis loads at the knee. We estimate that many of the DOFs from the upper torso have been cycled hundreds of thousands of times without failure. 6 . J O I N T A N G L E M E A S U R E M E N T The soft tissue interface allowed the placement of rigid plates on the SenSuit™ user in a consistent and stable manner. HallJoint sensors were designed to be small and accurate and able to assume a variety of geometries. This chapter details how the sensors were connected to the plates in order to obtain the most accurate joint angle measurements. 6 . 1 A p p r o a c h The most accurate joint angle measurements occur when the sensor center of rotation (COR) is aligned with the COR of the underlying joint. In reality, anatomical joints do not have simple CORs; rather, a joint will have an instantaneous screw axis that moves as the joint moves. We are not interested in measuring the motions about the screw axis and are more interested in how the motion of a limb appears to an external observer. For this reason, it is useful to assign a "center" to the joint, about which all motion of the joint occurs. It is this center with which we try to align our sensors. Assuming a well fitting soft tissue interface, the mechanism (and thus sensor) then faithfully follows the motions of the joint in a linear fashion. It would have been desirable to align all the sensors with their underlying joints; unfortunately, this was only possible at the wrist, elbow, knee, and (to a degree) neck joints. When the sensor and joint are not aligned, nonlinearities arise between the angle of the mechanism and the real angle of the joint. The mechanism nonlinearities are in addition to any nonlinearities inherent to the sensor. It is difficult to align sensor CORs with joint CORs for the more complex joints. The shoulder, hip, ankle, neck, and waist can be modeled as ball joints, each having 71 three perpendicular intersecting DOFs (in reality, only the hip joint is close to this model). A mechanism aligning sensors with the three orthogonal joint axes would need to surround the joint. Complications arise with this type of design because the sensors must be rigidly linked to each other, linked both proximal and distal to the joint, move with the joint, stay out of its workspace, and not collide with the body. Although this was done on the shoulders of the third SenSuit™ (and attempted on this version), it is exceedingly complex. The advantages of aligning the sensors (reduced nonlinearities) did not justify the disadvantages (increased weight, bulk, and complexity; an awkward singularity in the front of the workspace). A simpler way of measuring an anatomical ball joint is with a three DOF spherical joint. A spherical joint has three sensors with intersecting CORs arranged so that it can measure any orientation. Of course, it is impossible to locate the sensored joint within the human joint: it must be located outside the joint and skin. This prohibits aligning the sensors with the joint axes and introduces the nonlinearities described above. Another type of nonlinearity, coupling, also arises with the use of an external three DOF joint. Coupling is the movement of one or more DOFs during a pure movement of a different DOF. For instance, during pure shoulder rotation, the flexion and abduction sensors also move. Their motion is coupled to the motion of rotation. Joint nonlinearities and coupling can be reduced by locating the sensored joint as close as possible to the anatomical joint and by increasing the length of the link that connects the sensors to the limb. When physical improvements to the SenSuit™ structure failed to reduce nonlinearities to acceptable levels, transformation algorithms were written to compensate for the errors. This work is detailed in Chapter 7. The following sections explain the mechanisms used to measure each joint. For the neck, elbow, knee, and wrist joints, it was possible to align the sensor COR with the joint COR. For the rest of the joints, the sensor COR was placed close to the joint, positioned to reduce coupling and stay out of the limb workspace. 72 6 . 2 H e a d / N e c k Figure 6-1 shows that the neck is actually a column of seven cervical vertebrae grouped into the superior and inferior segments (31). The superior segment has three DOFs for each vertebrae, whereas the inferior segment allows for flexion and combined lateral/rotational movements for the other five vertebrae. Clearly, the neck consists of a complex series of joints that would be difficult to measure. Therefore, the neck model was simplified into a three DOF ball joint connecting the head to the torso. This allows the equivalent measurement of head flex/extension, ab/adduction. and rotation. More complex head and neck motions, such as bending the neck forward while keeping the head level, are impossible to measure Figure 6-1. The cervical vertebral column. (A) Superior segment, containing the atlas and axis (CI, C2). containing vertebrae C3-C5. Adapted from (31). (B) Inferior segment. 73 with this scheme. However, we have the advantage of a simple mechanism that measures the primary motions of the neck. Assuming that the neck has only three DOFs, the CORs were arbitrarily located halfway up the neck. This way, an "average" of the cervical vertebral movements were measured. The sensors were arranged so that their axes would align with the three virtual axes of the neck. This design was adopted from the third version SenSuit™. The mechanism for measuring neck movement begins at the torso links, where the head flexion/extension sensor is located (see Figure 6-2). A "U" shaped link connects the flexion sensor on the side to the abduction/adduction sensor located behind the neck. The width of the U was widened compared to the third SenSuit™, to allow the user to make larger lateral head movements. A right angle link connects the abduction sensor to the rotation sensor on top of the head. The rotation sensor is rigidly mounted to the head soft tissue attachment. 6 . 3 E l b o w / K n e e The elbow and knee joints are quite similar. A large proximal bone (the humerus or femur) connects to two distal smaller bones (ulna/radius or tibia/fibula) at the joint. The joint allows for two DOFs. flexion and rotation (see Figure 6-3). The smaller bones rotate around the larger bone in the case of flexion and rotate about each other in the case of rotation. Since the rotation is distal to the joint, it manifests itself in the wrist or ankle. The flexion DOF will be discussed here, and the rotation DOFs will be discussed in the wrist and ankle sections. Because these are fairly simple joints, the flexion sensors could be aligned with the joint COR. We assume that the rotation is pure, although it is known that the knee instantaneous center of rotation (ICOR) moves during flexion (3). This will cause small errors in link lengths and joint offsets. The proximal and distal plates of the joint were connected together by the sensor at the lateral side of the joint (see Figure 6-4 and Figure 6-2. Head DOF measurement. (A) Third SenSuit™. (B) Current SenSuit™. 75 Figure 6-3. The elbow and knee joints. Adapted from (2, 3). Figure 6-5). The third SenSuit™ design also tried to line up the CORs but failed because of plate drift due to its single plate design. It is interesting to note that the user can feel when the sensor is not closely aligned with the joint COR. A misalignment will cause the plates to move on the user's skin, making it easy for them to properly adjust the assembly. 6 . 4 W r i s t When the lower arm is viewed as a whole, the wrist appears to have three DOFs: flex/extension, ab/adduction, and rotation (commonly known as supination/pronation). Rotation does not occur in the wrist joint, rather, it occurs because the radius and ulna rotate about each other. Since this motion comes after the elbow, it appears that the wrist has a rotation DOF. Figure 6-6 shows the location of the CORs for the wrist. The wrist itself contains two joints. The radio-caipal joint, located between the radial head and the proximal end of the first row of carpal bones, allows for abduction 77 A B Figure 6-5. Knee DOF measurement. (A) Third SenSuit™ (B) Current SenSuit' 78 Figure 6-6. Wrist CORs. Adapted from (2). and adduction. The midcarpal joint, which allows for flexion and extension, is located between the proximal and distal rows of the carpal bones (2) (see Figure 6-7). The third SenSuit™ wrist was designed so that the flexion and adduction sensor CORs aligned with the wrist CORs. Rotation was measured via a flexible shaft connecting the distal elbow plate to the wrist plate. The flexible shaft allowed the plates to change position without creating a change in sensed rotation. This ensures that only axial rotational differences between the plates are measured. A slider at the wrist plate allows for different lengths of arms and for the shortening of the flexible shaft in the axial direction after a rotation. These mechanisms for measuring the wrist DOFs worked so well in the third SenSuit™ that they were essentially unchanged for the current SenSuit™ (see Figure 6-8) . 79 Figure 6-7. Joints in the wrist. (A) Radio-carpal joint, allowing for ab/adduction. (Al) Ab/adduction axis. (B) Mid-carpal joint, allowing for flex/extension. (BI) Flex/extension axis. Adapted from (2). 6 . 5 W a i s t The vertebral column is made up of 17 vertebrae, excluding the neck. The joint between each vertebrae has 6 DOFs: compression, flex/extension, ab/adduction, rotation, shear in the sagittal plane, and shear in the frontal plane. Even though each of these motions is small, they add up along the length of the spine to make the gross motions of the back. As in the head, it would be very difficult to measure all of these motions. For our animation purposes, three DOFs sufficiently represent the motions of the spine. Therefore, the SenSuit™ only measures three DOFs for the waist. The spine is assumed to be two links, joined at the waist by a ball joint. The motion of the upper torso is measured relative to the pelvis (see Figure 6-9). Several ideas w?ere proposed for measuring the waist DOFs. String potentiometers, used on the third SenSuit™, were low profile and fairly simple (see 80 81 Torso link Ball joint Pelvis link Figure 6-9. The vertebral column. Shown is the imaginary linkage that approximates spinal movements. Adapted from (31). Figure 6-10). However, they were difficult to calibrate and suffered from severe coupling. In addition, a user with a large belly would cause the strings to behave in an inconsistent manner. Older SenSuits™ had the waist sensors mounted at the back. We also decided to mount at the back of the current SenSuit™, in order to avoid the "belly" factor and to keep the mechanism out of the user's way. At first, a three DOF cluster was mounted at the torso and connected by a slider to the hip plates. This measured abduction and rotation fine, but flexion produced only small motions in the sensor (see Figure 6-11). For calibration, this required that the flexion gain be set very high, and any small motion 82 Figure 6-10. Third SenSuit™ waist measurement. Three strings crossed the front and side of the waist between the torso and hips (emphasized for clarity). The strings were connected to potentiometers that measured their length. The lengths were calibrated to waist angle measurements. of the torso or hip plates caused large disturbances in the waist measurement. We decided not to mount the flexion sensor at the hip plates, because it would have required a significant redesign of the waist sensor assembly. A simpler solution existed as a linear potentiometer mounted alongside the slider (see Figure 6-12). Thus, the potentiometer could measure the change in length of the slider during flexion, and this was calibrated to be a change in waist angle. Another problem was the abduction sensor mounted at the torso plate. For some people, it measured well, but others caused it to move very little, despite their obvious tilting. It turned out that some people tilted from high up in their spine, whereas others tilted low in their spine. The high tilters produced reliable data, while the low tilters did not. The abduction sensor was moved to the hip plate to accommodate the majority of our users (see Figure 6-12). A revision of the waist sensors in the future will likely include top and bottom abduction sensors, whose output will be summed. 83 Flexion sensor 7 S AS « AA A L = AA Figure 6-11. Failed three DOF waist sensor viewed from side. Flexion movements caused very little change in the flexion sensor, requiring large gains. However, a length sensor could be calibrated to measure waist flexion with a relatively small gain. Unfortunately, the waist design suffered from sensor coupling. During a pure rotation, both flexion and abduction sensors moved. The coupling was severe enough that a software algorithm to decouple the sensors was written, which is discussed in Chapter 7. The shoulders, hips, and ankles are each three DOF joints. Because of their geometry and ROM, designing mechanisms that align sensors to the joint axes is very difficult. Instead, the joint DOFs were measured with a three DOF cluster of sensors, offset from the anatomical joint. Although this introduced nonlinearities and sensor coupling, these were not very severe in the hips and ankles. The shoulder, however, had to be compensated for with a software algorithm (also discussed in Chapter 7). Each three DOF sensor cluster is connected at one end to a plate, which is 6 . 6 T h r e e D O F C l u s t e r s 84 Figure 6-12. Waist DOF measurement for the current SenSuit™. attached to the user. The first two DOFs (those closest to the plate) measure flex/extension and ab/adduction, and the third measures limb rotation and is connected to a slider. All three DOFs have intersecting CORs. The slider allows the SenSuit™ to fit users of different body proportions and changes length during limb movements (because the sensor CORs do not line up with the joint CORs). The other end of the slider snaps into the next plate on the chain. The snap between the slider and plate allows motions in flex/extension and ab/adduction, again to accommodate different body shapes and link movement. However, the snap is rigid in rotation, thereby transmitting the rotational movement of the limb, through the parallel slider, to the rotation sensor. 85 6 . 6 . 1 Shoulder The shoulder is actually a complex of five joints which form two groups. The first group (the shoulder joint) allows the movement of the humerus relative to the shoulder, whereas the second group (the shoulder girdle) is responsible for the movement of the shoulder relative to the torso. The shoulder has three DOFs: flex/extension, ab/adduction, and rotation. The shoulder girdle movements are complex but can be simplified to ab/adduction (vertical shrugging movements) and rotation (forward and back movements of the shoulder girdle) (see Figure 6-13). For our purposes, the shoulder girdle was not important. The robot anthropomorphic figures had no ability to move the shoulder girdle, and convincing computer animation can still be achieved without it. Therefore, no mechanism was designed to measure shoulder girdle movements. However, to get reliable data for the shoulder joint, the motion of the shoulder girdle had to be accounted for. This Shoulder flex/extension Shoulder girdle ab/adduction Figure 6-13. Shoulder CORs. Adapted from (2). 86 mechanism was explained in Chapter 4. The shoulder is a ball-and-socket joint (see Figure 6-14). The head of the humerus is roughly spherical, and joins the glenoid cavity on the lateral edge of the scapula. A ring of cartilage called the glenoid labrum surrounds the glenoid cavity, making the cavity deeper. However, the head of the humerus is still much larger than the glenoid cavity, making it a very open joint. This provides greater ROM at the expense of lessened joint stability (2). Because the head of the humerus is not spherical, the COR of the shoulder moves as the shoulder moves. This ICOR moves continuously during flex/extension and rotation but may move discontinuously between two locations during ab/adduction (see Figure 6-15) . Instead of trying to track the ICORs in the shoulder, we assumed that they were positioned in the center of the humeral head. Like the knee, this would result in small link length and joint offset errors. The designers of the third SenSuit™ tried to align the Humeral head Glenoid labrum Glenoid cavity Figure 6-14. The shoulder joint. Adapted from (2). 87 Figure 6-15. Shoulder ICORs of the head of the right humerus. The gray circles represent the continuous bounds of the ICOR. (A) Abduction. For the first 50° of abduction, the ICOR is located within the lower circle. For abduction greater than 50°, the ICOR is located within the upper circle. (B) Flexion. (C) Rotation. Adapted from (2). 88 shoulder sensors with the CORs in the shoulder, using a mechanism that began dorsal to the shoulder measuring ab/adduction, had an "L" shaped link that connected it to the flexion sensor lateral to the shoulder, and linked via a virtual center (measuring rotation) to the elbow (see Figure 6-16) . Although this shoulder design works in theory, with each sensor measuring a single DOF, in reality it suffered from a number of problems. As explained in Chapter 4, the shoulder plate of the third SenSuit™ was prone to movement and drift, which would misalign the sensors. The linkage had three DOFs, but they were only constrained at the Figure 6-16. Third SenSuit™ shoulder mechanism. 89 back of the shoulder and at the elbow. The loose constraints of the linkage and its inertia allowed the linkage to move independently of the shoulder. The linkage also had difficulty following the shoulder motion in some areas of the workspace. The mechanism still suffered from sensor coupling, the problem it was designed to solve. In addition, a sensor singularity in the front of the workspace caused unpredictable motions whenever the arms pointed forward. We tried making the new shoulder mechanism with aligned sensors, similar to the third SenSuit™. It was believed that by using lighter materials and connecting the linkage at the upper arm instead of the elbow, we could overcome the problems of the third SenSuit™. Unfortunately, the changes made no difference. The linkage was still unstable, and sensor coupling still occurred. The sensor alignment concept was abandoned for the shoulder. The new shoulder attachment was designed (see Chapter 4), and a simpler approach was attempted. A three DOF cluster was anchored to the shoulder attachment at the acromion, above and lateral to the joint, and linked to a slider that connected to the proximal elbow plate (see Figure 6-17). This way, it avoided the arm for all movements within the mechanism ROM. This design was similar to the second SenSuit™ shoulder mechanism (see Figure 6-17). However, the older mechanism had a poor attachment, had poor sensor positioning, and had severe coupling, leading the designers of the third SenSuit™ to abandon this approach. The current design benefits from a drift resistant soft tissue interface at the acromion and a location that reduces coupling and avoids the arm. 6 . 6 . 2 Hip Like the shoulder, the hip is also a ball-and-socket joint; however, because of its closed nature, it has a more limited ROM for its three DOFs: flex/extension, ab/adduction, and rotation (see Figure 6-18). 90 Figure 6-17. Shoulder DOF measurement. (A) Second SenSuit™. (B) Current SenSuit™. 91 Figure 6-18. Hip CORs. Adapted from (3). The head of the femur fits into the acetabulum of the pelvis, forming the hip joint. The femoral head is spherical, and the CORs intersect it at its center (see Figure 6- 19). The hip is covered by much more muscle and fat than the shoulder. This would require a very large mechanism to align sensors to the hip axes. Like the shoulder, a three DOF cluster was used instead, located laterally and above the actual hip joint (see Figure 6-20). The mechanism is similar to the third SenSuit™ hip sensors. The cluster is attached via a spacer to the hip plate at one end, while the other end has a link that connects it to the proximal knee plate. The spacer moves the link out laterally, to avoid the curvature of the upper leg. The lateral location of the cluster ensures that flex/extension movements of the leg will not collide with the sensors, while the vertical position allows for significant leg abduction without collision. 92 Figure 6-19. The hip joint. Adapted from (3). 6 . 6 . 3 Ankle/Foot The ankle and foot have three DOFs: flex/extension, ab/adduction (also known as supination/pronation), and rotation (also known as ab/adduction) (see Figure 6-21). Flex/extension occurs at the ankle joint, between the distal end of the tibia and fibula and the superior aspect of the talus. The foot bones have two joints, the subtalar joint (between the inferior surface of the talus and superior aspect of the calcaneus) and the transverse tarsal joint (between the anterior surface of the calcaneus and the posterior surfaces of the navicular and cuboidal bones) (see Figure 6-22). Together, they make an articular complex capable of the movements of inversion and eversion. Inversion is adduction of the foot accompanied by medial rotation and a slight extension. Eversion is the opposite motion of inversion. Foot motion is limited to these coupled in/eversion 93 Figure 6-20. Hip DOF measurement. (A) Third SenSuit™. (B) Current SenSuit™. Figure 6-22. Bones in the ankle and foot. Adapted from (3). 95 movements. Independent rotation is possible through the tibia and fibula, which can rotate about each other at the surface of the knee. This is perceived as rotational movement at the foot (3). The third SenSuit™ had a three DOF cluster located laterally to the ankle joint and connected to the foot via an aluminum stirrup. However, the stirrup was very uncomfortable and prone to extraneous movement due to its size and weight. Lateral placement of the sensors allowed them to hit objects in the environment, causing damage to the joint. The new three DOF cluster was placed above the foot on a slightly curved plate. This arrangement is far more comfortable and protects the sensors. The rotation sensor was connected via a sliding link to the distal knee plate (see Figure 6-23). 6 . 7 S u m m a r y The soft tissue interface and HallJoints work together to accurately measure joint angles. When possible, the CORs of the sensors and the joint were aligned. This gave the most accurate representation of the joint angle. Ball joints, however, presented a difficult design problem when the CORs were to be lined up. Although this was possible, and even tried, there were still significant problems with this approach. Instead, three DOF clusters of sensors were used to measure the ball joints. They were placed as close as possible to the joint, yet outside the limb workspace. The external placement of the three DOF sensors introduced nonlinearities and coupling into the joint angle data. For the shoulder and waist, the coupling was severe enough to require software decoupling algorithms to be written, which are discussed in detail in the next chapter. 7 . E L E C T R O N I C S / S O F T W A R E Our application requires that the SenSuit™ be able to communicate with a computer. Our current software is used to interpret SenSuit™ data, calibrate them, transform them, visualize them, and output them to another device such as a robot. This chapter describes the signal path from a raw sensor angle measurement to a digital representation of the user. 7 . 1 S e n S u i t™ E l e c t r o n i c s The interface between the SenSuit™ sensors and the computer is the Electronics Box (E-Box). The E-Box was created by SRC for previous SenSuits™ that used potentiometers. However, since the output of both a potentiometer and a HallJoint is a voltage dependent on angle, the E-Box is compatible with the new SenSuit™. The E-Box is located on the back of the torso harness, between the shoulder blades (see Figure 7-1). The location between the shoulder blades ensures that the E-Box will stay out of the user's way. Also, since cables are routed behind the arms and legs, there is little chance of snagging the cables while wearing the SenSuit™. Six connectors attach to the E-Box. Five connectors supply the sensor inputs from the various limbs: head, left arm, right arm, left leg, and right leg. Sensor data from the torso arrive at the E-Box through a forked cable coming out of the left leg connector. The sixth connector is the output line which sends the conditioned data to the computer serial port. Figure 7-2 shows how data from the sensors are processed by the E-Box before being sent to the computer. First, each HallJoint requires three wires: power (+5 V), Figure 7-1. E-Box location on the SenSuit™. ground, and output (range: 2-3 volts). These wires are routed to a connector on the E-Box along with other wires from the same limb. The signals are multiplexed by an analog multiplexer, and an amplifier stage gives the signal a separate gain and bias for each DOF. Normally, the gains and biases for all DOFs are set to 1 and 0, respectively. An analog to digital converter then digitizes the amplified values, which are finally processed by the HC11 microcontroller to allow for serial communication with the computer. 99 S e n S u i t H a i l j o i n t s Multiplexer Figure 7-2. Dataflow through the E-Box. The SenSuit™ has 34 measured DOFs. In the current configuration, the E-Box can output data at 30 Hz for a maximum of 48 DOFs. This allows up to 14 additional sensors to be added to the SenSuit™, without affecting the update rate. The maximum output capability of the E-Box with 48 DOFs is 100 Hz. 7 . 2 C a l i b r a t i o n P r o c e d u r e s When a user puts the SenSuit™ on and starts the visualization program, the initial correlation between the user's movements and the avatar may be quite poor. The gains and biases that map the sensor movements to the avatar's movements may not be appropriately set. The process of calibration insures that the gains and biases will be customized to the user, which will produce the best correlation between their movements and that of the avatar's. All of the following procedures involve a simple linear interpolation between two angles. 100 7 . 2 . 1 Calibration for Avatar without Transformation A l g o r i t h m s The simplest calibration procedure involves calibrating the avatar icon without using the linearizing transformation algorithms. It assumes that the SenSuit™ sensor angles are exactly linear with respect to the anatomical joint angles. This procedure uses a special subroutine within the graphical interface and requires the person to move his or her limbs into predefined poses. The entire suit can be calibrated this way in less than 3 min. Calibration begins in the graphics program by clicking on the "Calibrate" button. The computer then asks the user to position their limbs in a series of poses. The poses are designed in such a way as to measure each joint in two unique but reproducible angles. For instance, the first pose for the elbow would be with the arm straight. The computer would assign this position 0°. Bending the elbow to a right angle would be the second pose. The computer would then assign this position 90°. The computer then fits a line through the data. The angle of the limbs can be verified visually for a coarse calibration by using simple, right angle poses or by using an external "square" to verify the 90° angle. 7 . 2 . 2 Calibration for Avatar with Transformation A l g o r i t h m s If the transformation algorithms are to be used, then a modified version of the above calibration must be done. This calibration assumes that certain sensors have nonlinear relationships with their respective joints (the shoulders). For those joints, the sensor angles are calibrated rather than the limb (see Figure 7-3). Otherwise the calibration is the same as above. The reason sensor angles are calibrated is so that the linearizing algorithm has the most accurate data possible concerning the angles of the sensors. Then, it uses the 101 Figure 7-3. The difference between the limb angle and sensor angle. The limb angle is at 0°, but the sensor angle is at -15°. calibrated sensor data to compute the real angle of the limb. If the limb angle was calibrated, the true sensor angle would be misrepresented, and the algorithm would fail to compute the correct orientation of the limb. The values of the default angles used in this calibration came from a study of five individuals. The average height of the group was 70.2 inches, with a maximum of 73 inches and a minimum of 68