Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
insideOut: Estimating joint angles in tendon-driven robots using Artificial Neural Networks and non-collocated sensors
(USC Thesis Other)
insideOut: Estimating joint angles in tendon-driven robots using Artificial Neural Networks and non-collocated sensors
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
insideOut: Estimating Joint Angles in Tendon-Driven Robots Using Artificial Neural Networks and Non-collocated Sensors by Daniel A. Hagen A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY BIOMEDICAL ENGINEERING December 2020 Copyright 2020 Daniel A. Hagen ...the assumption that changes in muscle fiber length mimic changes in musculotendon length may be invalid, especiallyforhighlycompliantactuators, and demands careful scrutiny. — Felix E. Zajac ii To my amazing wife, Ali. My rock, my partner, my everything. I wouldn’t be here without you, nor would I want to be. iii Acknowledgements First, I would like to thank Dr. Francisco Valero-Cuevas, my primary advisor, for his continued guidance and support throughout my PhD work. I was first inspired to study tendon-driven systems while taking his graduate course. And after interning in his lab, I not only knew what I wanted to focus my PhD on, but who I wanted to work with. He fostered many of my academic tangents (no matter how tangential) on the quest to understand redundant tendon-driven control, and he encouraged collaborations between many different fields to provide thorough perspective. His lab lies at the intersection of math, physiology, robotics, and control theory, and I felt right at home there. I would also like to the rest of my committee members – Drs. Gerald Loeb and Mihailo Jovanovic. Time and time again Dr. Loeb has provided careful criticism for this work, but always did so in a constructive way so that I left our conversations with a better understanding of why and how things worked. He taught me the value of understanding the micro so that we might explain the macro. Dr. Jovanovic was instrumental in providing me with the control theory foundation needed in this work. He also was a tremendous sounding board during the early stages of my research when I was struggling to find my academic voice. To all of the members of my committee, I say thank you for your trust, support, criticism, and instruction. I am fortunate that our paths crossed, but I am eternally grateful that I actually got to work with you. I am also fortunate to come from a lab that thrives on collaborations. There are many lab members that I have interacted and collaborated with throughout my Master’s and PhD work, but I am especially grateful to Dr. Christopher Laine, a former postdoctoral fellow, and Dr. Akira Nagamori, a PhD graduate of the lab, for the countless hours we spent in front of a whiteboard, workshopping research ideas. It was there that I was able to critically think through research questions and somehow manage to turn a rant into a plan of action. I am also incredibly thankful for the collaboration with Ali Marjaninejad, a fellow PhD candidate from the lab, who opened my iv eyes to the world of artificial intelligence. Without the foundation that he built this work would not be possible. I am equally grateful to all the other unnamed lab members who helped shape my research. I will cherish our cathartic meetings and the friendships they help build. Lastly, I would like to thank both my immediate and extended family for their unwavering support. No one supported me more than my wife, Ali. Thank you for encouraging me to go back to school and for supporting us while I did. Your strength made all of this possible and I am forever grateful for your love and support. And while I’m at it, I guess I should also thank my 1.5 year old son, Carl, for being such an amazing sleeper and an overall great kid. This is for you too, bud! v TABLE OF CONTENTS Epigraph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Dedication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 0.1.1 Why study compliant tendon-driven systems? . . . . . . . . . . . . . . . . . 2 0.1.2 Statement of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 0.2 Significance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 0.3 Innovation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 0.4 Thesis Outline. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Chapter 1: Similar movements can elicitdifferent musculotendon states . . . . . . . . . . 9 1.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.3 Material and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3.1 Arm kinematic model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3.2 Defining initial and final arm postures, hand positions and velocities . . . . . 13 1.3.3 Generation of multiple valid hand trajectories . . . . . . . . . . . . . . . . . 17 1.3.4 Approximation of normalized muscle fascicle velocities . . . . . . . . . . . . 18 1.3.5 Definition of movement phases . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.3.6 Calculation of eccentric and concentric contraction velocity costs . . . . . . 22 1.4 Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.5 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Chapter 2: Tendon elasticity decouples kinematic states from actuator states . . . . . . . 32 2.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.3 Material and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.3.1 Pendulum Dynamics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.3.2 Tendon Tension Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.3.3 “Muscle” Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.3.4 Integrator Backstepping Constraint. . . . . . . . . . . . . . . . . . . . . . . 41 2.4 Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 2.4.1 Fixed Initial Muscle Fascicle Lengths . . . . . . . . . . . . . . . . . . . . . 51 2.4.2 Fixed Initial Tendon Tensions . . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.5 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 vi Chapter 3: Accurate musculotendon modeling requires tendon tension... sometimes . . . 62 3.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 3.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 3.3 Material and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 3.3.1 Derivation of Tendon Deformation as a Function of Tendon Forces . . . . . . 66 3.3.2 Error in Fascicle Length Approximations. . . . . . . . . . . . . . . . . . . . 67 3.3.3 New and Improved Equations for Musculotendon Length and Velocity . . . . 70 3.3.4 Definingc T andk T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 3.3.5 Defining Physiological Ranges forc T &k T . . . . . . . . . . . . . . . . . . 79 3.4 Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 3.4.1 Error from Ignoring Fascicle Pennation . . . . . . . . . . . . . . . . . . . . 80 3.4.2 Error from Assuming Inextensible Tendon . . . . . . . . . . . . . . . . . . . 85 3.5 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Chapter 4: Tendon tension enables better posture prediction in tendon-driven systems . . 94 4.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 4.3 Material and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 4.3.1 Definition of the Plant and its Dynamics . . . . . . . . . . . . . . . . . . . . 99 4.3.2 Tendon Stiffness Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 103 4.3.3 Description of Motor Babbling . . . . . . . . . . . . . . . . . . . . . . . . . 106 4.3.4 Training & Testing Artificial Neural Networks . . . . . . . . . . . . . . . . . 108 4.3.5 Defining Different Movements to Test Generalizability. . . . . . . . . . . . . 111 4.3.6 Controlling for Joint Angle and Stiffness via Feedback Linearization . . . . . 116 4.3.7 Description of Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 4.4 Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 4.4.1 Sweeping Motor Babbling Duration . . . . . . . . . . . . . . . . . . . . . . 122 4.4.2 Sweeping Number of Hidden Layer Nodes . . . . . . . . . . . . . . . . . . . 127 4.4.3 Neural Network Performance Across Sensory Sets and Movements . . . . . . 130 4.5 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 Chapter 5: Parameter Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 5.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 5.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 5.3 Material and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 5.3.1 Sweeping Movement Frequency. . . . . . . . . . . . . . . . . . . . . . . . . 144 5.3.2 Sweeping Tendon Stiffness and Motor Damping . . . . . . . . . . . . . . . . 145 5.3.3 Very High Tendon Stiffness Experiment. . . . . . . . . . . . . . . . . . . . . 150 5.4 Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 5.4.1 Sweeping Movement Frequency. . . . . . . . . . . . . . . . . . . . . . . . . 152 5.4.2 Sweeping Plant Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 5.4.3 Very High Tendon Stiffness Experiment. . . . . . . . . . . . . . . . . . . . . 161 vii 5.5 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 A Integrator Backstepping Example Source Code . . . . . . . . . . . . . . . . . . . . 195 B Addressing the Limitations of the “Parallelogram” Muscle Assumption. . . . . . . . 195 C insideOut Algorithm Source Code . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 viii List of Tables 2.1 Pendulum parameters. Values were chosen to approximate the dynamics of a hu- man forearm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.2 Musculotendon (MT) parameters used in the tendon tension dynamics (Eq. 2.6d). Maximum isometric tetanic muscle force (f max;i ), muscle pennation angle ( i ), and optimal tendon length (l T;o;i ) were taken from reported values for the biceps and triceps musculotendons (MTs) (combining information about different heads of the muscle as necessary, Holzbaur et al., 2005). Tendon shape coefficients (c T and k T ) were taken from Brown et al. (1996). . . . . . . . . . . . . . . . . . . . . . . 39 2.3 Muscle parameters used by Eq. 2.7. Muscle masses (m i ) and optimal muscle fascicle lengths (l m;o;i ) were taken from reported values for the biceps and triceps glsplMT (combining information about different heads of the muscle as necessary, Holzbaur et al., 2005). Muscle damping (b m;i ) was chosen to be small. . . . . . . . 40 2.4 Muscle parameters used by Eqs. 2.8–2.11. All coefficients are taken from Song et al. (2008b) from fast twitch muscles, except for L max CE , which was estimated as a floating parameter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.1 When calculating the MT excursionexcursion induced by the rotation of a joint, ignoring how much the moment arm (r i ) changes with respect to the joint angle ( i ) will result in an underestimate of how much the MT has either shortened (red) or lengthened (blue). Whether or not a given joint rotation will caused shortening or lengthening depends on the sign of moment arm (r i ( i )) as well as the sign of the change in joint angle ( i Valero-Cuevas, 2016). Therefore, the associated error in the MT excursion (" MT;i ) will have the same sign as the MT excursion and will be bounded by the intersection of this condition and Eq. 3.24 – which both change with the signs ofr i ( i ) and i . . . . . . . . . . . . . . . . . . . . . . . . 73 4.1 List of plant parameters for simple 1 degree of freedom (DOF) pendulum joint, actuated by two backdriveable motors that pull on nonlinearly compliant tendons. Tendon stiffness values (k T andb T ) were conservatively chosen to cause large, but reasonable tendon deformations for some maximum tension (See Section 4.3.2 for further explanation). Additionally, moment arm values (r j andr m ) were approxi- mated to reflect the overall scale of the pendulum. All other parameters were either taken or modified from those provided in Palli et al. (2008). . . . . . . . . . . . . . 102 4.2 List of variables for Eq. 4.28 that solve an algebraic Ricatti equation for continuous time. Derived in Palli et al. (2008) and reproduced here. . . . . . . . . . . . . . . . 118 ix 5.1 List of parameters chosen to describe tendon tension-deformation relationship (de- fined in Section 4.3.2 and reproduced in Eq. 5.4) and the relative stiffness (com- pared to the default value used in Chapter 4) for the experiments described in Sec- tions 5.3.2 (Parameter Sweep) & 5.3.3 (Very High Stiffness). Note that in order to make the minimum joint stiffness consistent across trials the value k T b T (i.e., the minimum tendon stiffness given by Eq. 5.4) was conserved for each choice of tendon stiffness parameters. (*) The approximate range of Young’s moduli values (E) were calculated from a conservative 0.001 m wide tendon with resting lengths between 0.01 and 0.04 m when under 400 N of tension. . . . . . . . . . . . . . . 148 x List of Figures 1.1 Schematic of our 18 muscle, 3 DOF model which considers sagittal-plane shoulder flexion/extension (F/E), elbow flexion/extension and wrist flexion/extension only. Forearm pronation/supination and radial-ulnar deviation were excluded from our model as studies have shown player’s to typically keep the shoulder, elbow, wrist, and ball in the same plane as the basket (Knudson, 1993). For an overview of the muscles used and their posture-dependent moment arm functions see Valero- Cuevas (2016), Ramsay et al. (2009), and Holzbaur et al. (2005). . . . . . . . . . 12 1.2 Calculation of the final hand endpoint velocity vector (~ v f ) necessary for a success- ful free throw. As this throw is well modeled as a ballistic problem, the indepen- dent variables are the angle of release () and the distance of the ball’s center from the basket’s center (x, y). (a) Utilizing the anthropometric geometric model described in Eq. 1.1, a player’s height and the joint angles at the point of release are utilized to find the position of the hand endpoint relative to the shoulder joint (designated here as the origin). (b) The radius of the ball and the parameters of the basket (i.e. height of the basket and horizontal distance of the shoulder joint to the basket’s center) are incorporated to find the necessary displacement parameters (x, y). (c) Utilizing these displacements and the angle of release (), ballistics equations are rearranged and used to solve for the necessary release velocity vector for a successful free throw. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.3 Overview of trajectory generation technique. (a) Initial and final arm postures (taken from averaged, sample measurements) were passed through the kinematic model to find the initial and final hand endpoint positions (~ x i and~ x f ) and twists ( _ ~ x i and _ ~ x f ) (See Figure 1.2 for overview of ballistics). (b) Initial and final angular velocities ( _ ~ i and _ ~ f ) were calculated from their relationship to endpoint twists by the inverse Jacobian matrix (J 1 ( ~ )) evaluated at their respective joint angles ( ~ i and ~ f ). (c) A random break point (i.e. knot or seed) was generated for each joint angle by uniformly sampling from the joint’s range of motion as well as from the time between initial and final postures (0-550 ms). (d) Time histories for each joint angle were generated using clamped cubic splines to create smooth, piece-wise polynomial trajectories with proper initial and final conditions. Steps (c)-(d) were repeated if a resulting trajectory exceeded the joint’s range of motion or if undesirable (and unrealistic) rotations or velocities were encountered. (e) Combining joint angle time histories resulted in angle-angle-angle trajectories in configuration space. (f) Passing these combined angle time histories through the geometric model (i.e. the forward kinematic model) generated a hand endpoint trajectory with appropriate initial and final positions and velocities. . . . . . . . . 17 1.4 20 trajectories in (a) configuration space and (b) endpoint space from a uniformly sampled solution space. These trajectories serve as examples of the solution space while individual trajectories will be explored further later in the analysis. . . . . . 18 xi 1.5 Normalized muscle fascicle velocities vs. time for a randomly selected free throw attempt both in its entirety (left) and bounded by5 optimal muscle fascicle lengths per second (l m;o =s) (right) to highlight the upstroke phase. Muscles that only cross the shoulder are shown in navy, while the biceps and triceps muscles are shown in orange. The remaining muscles that cross the elbow and/or wrist are shown in light blue. Note that there exist two major zeros crossings during which many of the muscles change the direction of their contractions. The first instance occurs during the upstroke where the wrist extends as elbow flexes with similar but opposite angular velocities–while the shoulder continues to rotate upwards causing the net MT excursion time derivative for the bi-articulating muscles of the elbow and wrist to offset each other. The second instance defines the start of the power stroke phase where either the angular velocities of the elbow and/or wrist create a net zero MT excursion time derivative or the muscles of the shoulder change the direction of their contractions (if they do at all). The muscles that only cross the shoulder do not exhibit this first zero crossing as they typically experience con- traction throughout the entire upstroke without a change in direction (i.e. constant upward rotation about the shoulder joint), but will often exhibit the second zero crossing as a result of the cubic spline algorithm and the local extrema generated in the shoulder angle trajectory. As the biceps and triceps muscles cross over both the shoulder and elbow joints, they do not experience a zero crossing during the upstroke phase (i.e. the direction of both rotations are consistent and nonzero dur- ing this phase), but they do, however, exhibit typical zero crossings at the start of power stroke phase brought on by a major change in the direction of elbow rotation. 20 1.6 Two dimensional histogram of the eccentric and concentric costs of all trajectories as defined by equations 1.14 and 1.15, respectively (in normalized units l m;o =s (Zajac, 1989)) with one dimensional histogram axes overlays. . . . . . . . . . . . 23 1.7 Two dimensional histogram of the eccentric and concentric costs of all trajectories as defined by equations 1.14 and 1.15, respectively (in normalized units l m;o =s) (bottom right) with sample trajectories 1, 2 and 3 plotted in configuration space (top right) and endpoint space (left). . . . . . . . . . . . . . . . . . . . . . . . . . 24 1.8 Endpoint space plots of sample trajectories 1, 2, and 3, overlaid with their 20 most similar trajectories, respectively, as per eccentric and concentric cost values (gray). 24 1.9 Sample trajectories 1, 2, and 3 shown in endpoint space (top) with their corre- sponding normalized muscle fascicle velocity profiles and joint angle trajectories (bottom 3 rows). Note that the dotted lines in the velocity plots indicate5l m;o =s, while dotted lines in the joint angle plots indicate the allowable range of motion for each joint. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 xii 1.10 (a) Histogram of the average residual per time step generated when comparing sample trajectory 1 to all remaining trajectories with cumulative distribution (solid black line) overlaid. Half of the trajectories had average residual values 14.60 cm while the mode of this distribution was 11 cm. (b) Endpoint space plot of the 100 most similar trajectories compared to sample trajectory 1 (green) as de- termined by the average residual per time step ( 1.96 cm). (c) The individual distributions (axes overlays) and the joint distribution of these most similar trajec- tories (in normalized unitsl m;o =s). . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.1 Example of the nullspace of an equation inR 2 and its relationship to a particular solution (red) given by some cost function (e.g., minimizing the Euclidean norm) and the homogeneous solution (blue) which lies in the nullspace (N) of the equa- tion (grey) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 2.2 Pendulum controlled by two Hill-type muscle-like actuators that pull on nonlin- early compliant tendons to produce tensions (f T;i ) that produce torques at the joint through their moment arms (r i ). . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 2.3 Posture dependent moment arm functions (r i ) for “muscle 1” (biceps) and “muscle 2” (triceps) provided by Ramsay et al. (2009). The elbow angle is measured from the anatomical position and shoulder angle is assumed to be constant. . . . . . . . 38 2.4 Example of the integrator backstepping approach to sampling the nullspace of a compliant tendon-driven system (A,B), made to follow a reference trajectory (C), by prescribing the activation of one actuator (D). The resulting states are shown in the bottom panel (E). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 2.5 Relationship between initial muscle fascicle lengths and initial muscle activations for a given initial equilibrium tendon tension. Note that for initial muscle fascicle lengths near the optimal muscle fascicle length the range of compatible initial acti- vations is quite small. However, when the muscle fascicle length becomes smaller (l m;i < l m;o;i ), the muscle is on the descending curve of its force-length relation- ship (F L ) which dominates the denominator causing larger required activations to produce the same tendon tension. Alternatively, when the muscle becomes in- creasingly stretched (l m;i > l m;o;i ), the passive elasticity of the muscle dominates (F PE 1 ), requiring less muscle activation to produce the same initial tendon tension. 51 2.6 Sweeping tendon pretensioning levels. For 10 different baseline tensions an in- tegrator backstepping controller was used to force a pendulum controlled by two muscle-like actuators that pull on compliant tendons to follow a sinusoidal trajec- tory. For this simulation, the initial muscle fascicle lengths were fixed for each trial while the initial tendon tensions were selected such that they satisfy Eq. 2.56 and 0:15f max;i f T;i 0:5f max;i fori2f1; 2g. Muscle lengths and velocities have been normalized by their respective optimal muscle fascicle lengths (l m;o;i ). . . . . 52 xiii 2.7 Error between reference trajectory ( r (t)) and the actual joint angle () (controlled via an integrator backstepping approach) when pretensioning is systematically var- ied. All 10 trials are superimposed on top of one another, so it is clear that the performance was not affected by the choice of initial tendon tension. For this sim- ulation, initial muscle fascicle lengths were fixed while the initial tendon tensions were selected such that they satisfy Eq. 2.56 and 0:15f max;i f T;i 0:5f max;i for i2f1; 2g. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 2.8 Comparing normalized muscle fascicle velocities to the MT velocities associated with the movement (normalized by optimal muscle fascicle lengths) for muscles 1 (blue) and 2 (red) for 10 trials with different levels of pretensioning. The error between these two measurements is explicitly plotted on the right. Clearly, the assumption that MT velocity can be used to approximate muscle fascicle velocity will incur tension and muscle specific errors. . . . . . . . . . . . . . . . . . . . . 54 2.9 Comparing normalized muscle fascicle length changes (with respect to the initial muscle fascicle length) to the MT excursions (i.e., length changes) associated with the movement (normalized by optimal muscle fascicle lengths) for muscles 1 (blue) and 2 (red) for 10 trials with different levels of pretensioning. The error between these two measurements is explicitly plotted on the right. Consistent with the data seen in Figure 2.8, increasing the level of pretensioning (and as a result, the average amount of tension on each tendon) appears to increase the magnitude of the error for muscle 1, but decreases the the magnitude of the error for muscle 2. Clearly, the assumption that MT excursions can be used to approximate muscle fascicle length changes (and by extension, muscle fascicle lengths themselves) may incur tension and muscle specific errors. . . . . . . . . . . . . . . . . . . . . . . . . . . 55 2.10 Plotting the mean absolute error between the (ground truth) muscle fascicle length changes and the MT excursions (expressed as a percentage of the optimal muscle fascicle length) as a function of the level of pretensioning on the tendons. These results are consistent with the trends seen in Figures 2.8 & 2.9 where we see a positive relationship between the error and the pretensioning for muscle 1, but a negative relationship for muscle 2. This relationship does not appear to be linear, but does appear to be well defined, implying that an explicit form of the error may exist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.11 Sampling initial muscle fascicle lengths. For 25 different initial muscle fascicle lengths an integrator backstepping controller was used to force a pendulum con- trolled by two muscle-like actuators that pull on compliant tendons to follow a sinusoidal trajectory. For this simulation, the initial tendon tensions were fixed for each trial while the initial muscle fascicle lengths were uniformly selected such that they satisfy Eq. 2.56. Muscle lengths and velocities have been normalized by their respective optimal muscle fascicle lengths (l m;o;i ). . . . . . . . . . . . . . . . 57 xiv 2.12 Error between reference trajectory ( r (t)) and the actual joint angle () (controlled via an integrator backstepping approach) when initial muscle fascicle lengths are uniformly sampled. All 25 trials are superimposed on top of one another, so it is clear that the performance was not affected by the choice of initial muscle fascicle lengths. For this simulation, initial tendon tensions were fixed while the initial muscle fascicle lengths were uniformly sampled subject to Eqs. 2.57-2.58. . . . . 58 2.13 Comparing normalized muscle fascicle velocities to the MT velocities associated with the movement (normalized by optimal muscle fascicle lengths) for muscles 1 (blue) and 2 (red). The error between these two measurements is explicitly plotted on the right. While there clearly exists a difference between MT and muscle fas- cicle velocities, it does not appear that any obvious relationship exists between the initial muscle fascicle length and this difference. . . . . . . . . . . . . . . . . . . . 58 2.14 Comparing normalized muscle fascicle length changes (with respect to the initial tendon tensions) to the MT excursions (i.e., length changes) associated with the movement (normalized by optimal muscle fascicle lengths) for muscles 1 (blue) and 2 (red). The error between these two measurements is explicitly plotted on the right. Consistent with the data seen in Figure 2.8, there are differences between MT excursions and muscle fascicle length changes, but the differences do not appear to depend on the choice of initial muscle fascicle length. . . . . . . . . . . . . . . 59 2.15 Plotting the mean absolute error between the (ground truth) muscle fascicle length changes and the MT excursions (expressed as a percentage of the optimal muscle fascicle length) as a function of initial muscle fascicle lengths. These results are consistent with the trends seen in Figures 2.8 & 2.9 for muscle 2, but we do see a slight positive trend for muscle 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.1 Approximation of MT geometry as a flattened parallel bundle of pennated muscle fascicles in series with tendon (A) such that the change in length or excursion (l MT , B) is defined as the sum of tendon length change (combining the tendons of origin and insertion, or l T = l T;1 + l T;2 ) and change in the portion of muscle fascicle length projected onto the line of action of the MT (l m (t) cos((t)) l m (t o ) cos((t o )), Gans and Bock, 1965; Gans, 1982; Zajac, 1989). . . . . . . . . 63 xv 3.2 Evolution of MT excursion equations (s) and their differences. The constant mo- ment arm equation (Valero-Cuevas, 2016; Kurse et al., 2012; An et al., 1983) is the simplest approximation but clearly the arc length (dashed purple) does not ac- curately convey the true MT excursion (black). This was extended in Hagen and Valero-Cuevas (2017) where the true arc length was approximated by integrating the posture-specific moment arm function (orange). Even for sufficiently small , this approach does not completely capture the true MT excursion as it ignores the change in moment arm with respect to the joint angle. Correcting for this, we find the true MT excursion from the equation for arc length in polar coordinates (green). As the true MT excursion relies on the Euclidean of the moment arm and its partial derivative, the error between the approximation proposed in Hagen and Valero-Cuevas (2017) and the true equation derived here can be bounded by the triangle inequality (see Eq. 3.23). . . . . . . . . . . . . . . . . . . . . . . . . . . 72 3.3 Physiologically realistic ranges forc T andk T under the assumptions that (i)c T k T < 0:20 (red, Brown et al., 1996), and (ii), by definition, when force is negligible, tendon length equals its slack length (i.e., ^ f T 0! ^ l T 1, blue, Proske and Morgan, 1987; Zajac, 1989; Brown et al., 1996; Magnusson et al., 2001). . . . . . 79 3.4 Contour map (left) for the percentage of MT excursion that would be incorrectly mapped onto the line of action of the muscle fascicles (C 1 ) as the result of assuming some constant pennation angle as a function of the true pennation angle. For any assumed value of constant pennation angle ( c ), the resulting plot ofC 1 is given by the corresponding horizontal cross-section of the contour plot. Examples of plots for lower and higher values of c are shown on the right. Note that the error is negligible when the true pennation angle is equal to the assumed value (diagonal line on the left, and zero-crossings on the right). Additionally, the error is less than 5% when the assumed and actual pennation angles are less that 18 . . . . . . 81 xvi 3.5 Sensitivity of the relative error coefficients C 1 (the proportion of MT excursion not projected back onto the line of action of the muscle fascicles, A) andC 2 (the proportion of the initial muscle fascicle length not projected back onto the line of action of the muscle fascicles, B) from Eq. 3.14. A small deviation (5 ) was applied to c or(t o ), respectively, and the resulting change in the coefficients were plotted. For (A), while the error is minimized when(t) = c , as c increases the same deviation from the true pennation angle ((t) = c 5 ) will produce larger changes in C 1 and, therefore, a larger percentage of l MT would be incorrectly projected back onto the line of action of the muscle fascicles. Similarly for (B), the error will be minimized when the pennation angle does not change from the initial value (i.e.,(t) =(t o )), but as(t o ) increases, the same deviation from the initial value ((t) =(t o ) 5 ) will result in larger changes toC 2 and, therefore, a larger proportions of the initial muscle fascicle length would be incorrectly mapped back onto the muscle fascicles at timet. Ranges of pennation angles reported in the literature have been provide for a few muscle groups for reference (Yamaguchi et al., 1990; Lieber et al., 1990; Lieber and Brown, 1992; Herbert and Gandevia, 1995; Fukunaga et al., 1997; Martin et al., 2001; Ward et al., 2009; Kwah et al., 2013). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 3.6 Contour map (left) for the percentage of initial muscle fascicle length that would be incorrectly mapped back onto the true line of action of the muscle fascicles (C 2 ) as the result of assuming some constant pennation angle as a function of the initial and current pennation angles. Regardless of the assumed constant pennation angle value, this error will depend on the amount by which the pennation angle changes from its initial value. Therefore, for some initial pennation angle, the resulting plot of the coefficientC 2 is given by the corresponding horizontal cross-section of the contour plot. Examples of plots for lower and higher values of (t o ) are shown on the right. Note that the error is negligible when the current pennation angle is equal to the initial pennation angle (diagonal line on the left, and zero-crossings on the right). Additionally, the error is less than5% when the initial and actual pennation angles are less that 18 . Therefore, for muscles with small pennation angles (< 18 ) that do not drastically change over the course of a movement, this error will be relatively small (<5%). . . . . . . . . . . . . . . . . . . . . . . . 84 xvii 3.7 Relative error in muscle fascicle length as a result of assuming inextensible ten- dons (Eq. 3.10c) that accounts for the previously ignored tendon length change, scaled by the tendon slack length to optimal muscle fascicle length ratio (l T;s =l m;o ), and projected back onto the line of action of the muscle fascicles. Note that the error will be zero when the tension of the tendon is equal to the initial tension ( ^ f i T (t o )—i.e., no net deformation of the tendon has occurred). Two different initial tension values have been chosen to demonstrate that starting at lower forces (i.e., an intercept towards the left) creates greater sensitivity to deviations from the initial tension, whereas at higher forces the sensitivity is lower and approaches the asymp- totic slope (black arrows) where it will be proportional tol T;s =l m;o and inversely proportional to c T = ECSA T =f max (i.e., the tendon’s normalized asymptotic stiffness). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 3.8 Parallel coordinates plot for the relative error in muscle fascicle length () associ- ated with assuming inextensible tendons for 10,000 random samples in the 5 pa- rameters of interest (c T ,k T ,l T;o =l T;s ,l T;s =l m;o , and) within their reported physio- logical ranges during random isometric force tasks (top). As MT excursion (l MT ) is zero during isometric contractions, assuming inextensible tendons is equivalent to assuming that muscle fascicle length is constant at a given percentage of max- imum voluntary contraction (MVC) and the error, therefore, will be identical to the normalized length change of the tendon, scaled byl T;s =l m;o andl T;o =l T;s , and divided by the cosine of the pennation angle. For higher forces ( 75% MVC), we find that high errors ( 30%l m;o ) can occur for all values ofc T andk T as, by definition, the tendon length converges to its optimal length (i.e., the error is only proportional tol T;o =l T;s andl T;s =l m;o , second from top). Alternatively, we find that the error can be equally large for lower forces ( 50% MVC) when the tendon has low stiffness, low curvature (high radius of curvature), and larger ratios ofl T;s to l m;o and l T;o =l T;s (i.e., the “perfect storm”, second from bottom). Conversely, if any of these conditions are not met, the errors in muscle fascicle lengths can be low (bottom). Lastly, pennation angles do not appear to preclude any muscles from this sort of error, but it is trivial to show that increasing will increase the propor- tion of l T projected back onto the line of action of the muscle fascicles. Visit https://daniel8hagen.com/images/tendon length change parallel coords to access this interactive parallel coordinate plot online. . . . . . . . . . . . . . . . . . . . 88 xviii 4.1 Example of types of biological sensors present in a muscle. From the spinal cord -motorneurons project to the muscle fascicles to activate it. From there, the muscle spindles are responsible for sensing the resulting muscle fascicle lengths and velocities (l m & _ l m , orange) while the Golgi tendon organs are responsible for detecting tendon tension (f T , blue). These sensory signals may be integrated through their spinal and supraspinal projections to form internal representations of expected or virtual limb position (; Scott and Loeb, 1994; Dimitriou and Edin, 2008; Van Soest and Rozendaal, 2008; Kistemaker et al., 2013). Note that there are additional biological sensors that detect stretch in the skin and synovial capsule not shown here, but these do not directly encode joint position either (Kandel and Schwartz, 2000). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 4.2 Schematic of tendon-driven system with 1 kinematic DOF and 2 degrees of ac- tuation (motors) that pull on tendons with nonlinear elasticity (creating a tension, f T;i ). The motors are assumed to be backdriveable with torques ( i ) as inputs. . . . 100 4.3 Proposed setup for training artificial neural networks (ANNs) on motor babbling. Random input torques were generated from low frequency, band-limited white noise (1-10 Hz) chosen such that the difference between the two signals has a nor- mal distribution (0 0:5 Nm). These motor babbling signals are passed through the plant and all subsequent sensory information is recorded. Lastly, an ANN is trained on a particular set of sensory information (~ x i sens ) to predict joint angle ( i j;pred ).107 4.4 Example of how 300 ms of motor babbling signals are generated. (A) Random motor torque inputs are uniformly sampled from the range of possible inputs and assigned to 50 ms windows for motor 1 (red). (B) Then values for motor 2 (blue) are selected for each window from a normal distribution centered around the values for motor 1 with a standard deviation of 0.5 Nm (5% of the range of maximum in- put level). These discontinuous, piecewise signals are then filtered using a forward (C) and backward (D) finite impulse response moving average filter with a filter lengths of 50 ms. This results in correlated band-limited, low frequency ( 10 Hz) white noise motor babbling signals. . . . . . . . . . . . . . . . . . . . . . . . 108 4.5 Example plots of four different types of reference trajectories where joint angle and joint stiffness are either varied sinusoidally or in a point-to-point task. For point-to- point tasks, transitions are limited to 0.25 seconds (2 Hz cutoff) and are designed in a way that they are continuously differentiable up to the fourth derivative (i.e., they leave and arrive each point with zero velocities, accelerations, jerks, and snaps). . 116 xix 4.6 Sample plots comparing the performance of the feedback linearization algorith when following the four reference trajectories of interest (where joint angle and joint stiffness are varied sinusoidal or with a point-to-point task). The resulting motor positions and velocities, tendon tensions, and motor input torques are pro- vided beneath the joint angle output of each trajectory. It should be noted that the motor kinematics are not always coupled to the behavior of the pendulum (espe- cially when the joint angle is constant during a point-to-point task but the joint stiffness is varied sinusoidally, middle right), suggesting it would be difficult to imply joint angle from motor measurements alone. . . . . . . . . . . . . . . . . . 120 4.7 Proposed experimental setup for a single choice of either babbling duration (Exper- iment 1,N = 25) or the number of hidden layer nodes (Experiment 2,N = 10). For either experiment, for each choice of the independent parameter, N motor babbling experiments are conducted and N ANNs are trained (See. Figure 4.3). The performance of each of these networks will be determined by their ability to generalize to different movements (where joint angle and/or stiffness are pre- scribed either sinusoidal (Sin) or point-to-point (P2P) trajectories; See Figure 4.5). A feedback linearization controller then calculates the input torques needed to pro- duce the desired movements (See Section 4.3.6), which are then passed through the plant to produce the experimental joint angle ( j;exp ) as well as the four sensory sets of interest (~ x i sens ). These sets are then passed through their corresponding ANNs that were trained on babbling data to predict joint angle ( i j;pred ). The prediction errors for each network are then averaged over all trials, and the performance as a function of the independent parameter can then be evaluated. . . . . . . . . . . . . 121 4.8 Plots of the average performance (mean absolute error) versus babbling duration (seconds) assuming 15 hidden layer nodes. For each babbling duration, 25 ANNs were trained from babbling data and the average error for each generalization tra- jectory was computed. The ANNs with tendon tension drastically outperform those trained only with motor information. A log scale is provided in Figure 4.9 to dis- cuss the performance of the Bio-Inspired Set relative to the set of All Available States. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 4.9 Plots of the average performance (mean absolute error) versus babbling duration (seconds) in log scale assuming 15 hidden layer nodes. For each babbling duration, 25 ANNs were trained from babbling data and the average error for each gener- alization trajectory was computed. The ANNs trained with tendon tension (dark blue and orange) drastically outperform those trained only with motor information (light blue and purple, with a 3 orders of magnitude improvement when training on 7.5 seconds or longer). We also see that the Bio-Inspired Set will on average perform as well as our baseline set (All Available States), suggesting that tendon tension in addition to motor positions and velocities are sufficient to predict joint angles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 xx 4.10 Plots of the standard deviation in the performance (mean absolute error) for each choice babbling duration (seconds) in log scale assuming 15 hidden layer nodes. For each babbling duration, 25 ANNs were trained from babbling data and the av- erage error for each generalization trajectory was computed. We see trends similar to those seen in Figure 4.9, with standard deviation generally decreasing as the du- ration of babbling increases. Note that because this plotted on a log scale, the peaks for both the All Available States and Bio-Inspired Set at 20 seconds of babbling are on the order of 10 3 to 10 2 , and therefore does not reflect large variations from the average values but more likely noise. . . . . . . . . . . . . . . . . . . . . . . 126 4.11 Plots of the average performance (mean absolute error) versus number of hidden layer neuron assuming 15 seconds of motor babbling. For each choice in the num- ber of hidden layer nodes, 10 ANNs were trained from babbling data and the av- erage error for each generalization trajectory was computed. For any choice in the number of hidden layer nodes, the ANNs trained with tendon tension outperform those trained without it. However, with fewer hidden nodes, the performance of ANNs trained without tendon tension improves while the performance of ANNs trained with tendon tension degrades but still perform best. As expected the perfor- mance of the ANNs trained with motor information only worsens as the structure of the ANN becomes more complex because there is not enough information to tune the additional weights. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 4.12 Plots of the average performance (mean absolute error) versus number of hidden layer neuron in log scale assuming 15 seconds of motor babbling. For each choice for the number of hidden layer nodes, 10 ANNs were trained from babbling data and the average error for each generalization trajectory was computed. For any choice in the number of hidden layer nodes, the ANNs with tendon tension out- perform those trained only with motor information. However, for fewer and fewer hidden layer nodes, the performance of the ANN trained without tendon tension improves while the performance of ANNs trained with tendon tension data degrade but still perform best. It can be seen that the performance of the ANNs trained on tendon tension data begin to plateau for 9+ hidden layer nodes. . . . . . . . . . . 129 4.13 Performance (root mean squared error; RMSE in degrees) versus the number of epochs needed to train each ANN. For each of the four sensory sets, 25 motor bab- bling simulation of 15 seconds were performed to train ANNs with 15 hidden layer nodes. Although it took the ANNs more than 1000 epochs for the performances to converge (even requiring up to 10,000 epochs for the ANNs trained on tendon tension data), the majority of the performance improvement came within the first 20–50 epochs (middle). In fact, the ANNs trained only on motor data (Motor Posi- tion and Velocity Only and All Motor States) converge with as little as 6 epochs (as seen by the average plot on the right). Lastly, it appears that learning from motor information may allow for faster learning, but the performance is soon beaten by the ANNs trained with tendon tension (which took longer to learn). . . . . . . . . 131 xxi 4.14 Average number of epochs used before training was terminated (bars: standard deviation). For each sensory set, 25 ANNs (with 15 hidden layer nodes) were trained on 15 seconds of motor babbling. As expected, it took longer to learn when using tendon tension (left two sets), as there were more features to extract from the data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 4.15 Bar plots of the average performance of each of the four ANNs (trained on the four sensory sets) when predicting joint angle from the four generalization movements (plotted on a log scale). For each sensory set, 25 ANNs with 15 hidden layer nodes were trained with 15 seconds of motor babbling (1 kHz sampling frequency) and their performance (mean absolute error) was averaged to compare across sensory sets and across movements. For each sensory set, there is little difference across movements, but there is a consistent trend that the sensory sets that include ten- don tension (All Available States and the Bio-Inspired Set) perform 3 orders of magnitude better than the sets trained without tendon tension. . . . . . . . . . . . 133 4.16 Radial bar plots for the log average performance for different joint angle bins (ev- ery 15 degrees) for each sensory set across all four generalization movements. The average performance for each ANN appears to be consistent across the joint angle space (i.e., there is no clear dependence on the actual joint angle and error from the predicted joint angle). While we again see that the ANNs trained with tendon ten- sion information (left two columns) outperform the ANNs trained without it (right two columns) by about 3 orders of magnitude, we now see that this is generally true across the entire joint angle space regardless of movement type. . . . . . . . . 134 4.17 Heatmap of the average mean absolute error versus joint angle and joint stiffness. It is clear that the ANNs trained on tendon tension (left two columns) can reliably predict joint angle at any level of joint stiffness, while the ANNs trained without tendon tension (right two columns) have difficulty at low joint stiffness values (re- gardless of the movement task). This is because at lower joint stiffness, the tendons are less stiff (i.e., more disproportionate lengthening per unit force) which causes more nonlinear decoupling between motor and kinematic states. . . . . . . . . . . 136 5.1 Examples of tendon tension deformation curves for the parameters chosen for the Parameter Sweep Experiment (red, Section 5.3.2) and the Very High Stiffness Ex- periment (blue, Section 5.3.3). The trace seen in solid red represents the force- length relationship for the compliant tendons used in Chapter 4. . . . . . . . . . . 149 xxii 5.2 Bar plot of the average performance (MAE) of each sensory set as function of the frequency of the sinusoidal joint angle trajectory. The joint stiffness was either var- ied sinusoidally or by a point-to-point task (Eqs. 5.1–5.2). The ANNs trained on tendon tension (left two sets) appear to generalize better to higher frequency move- ments, only worsening slightly when the movements becomes fastest. The ANNs trained on motor information only may decrease their performance by a similar or- der of magnitude, but there is quite a difference between producing average errors of 10 2 and 10 1 degrees. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 5.3 Radial bar plot of the average performance (MAE) of each sensory set as function of both the frequency of the sinusoidal joint angle trajectory and joint angle when the joint stiffness is varied sinusoidally as well (twice the frequency). The ANNs trained on tendon tension (top two sets) generalize better to higher frequency move- ments, only worsening slightly when the movements becomes fastest. For these two sets, when movements are the fastest, the largest errors appear to occur at the boundaries of the sinusoidal movement (which has interesting consequences to speed/accuracy trade offs). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 5.4 Radial bar plot of the average performance (MAE) of each sensory set as func- tion of both the frequency of the sinusoidal joint angle trajectory and joint angle when the joint stiffness is varied with a point-to-point task. Again, ANNs on ten- don tension (top two sets) generalize better to higher frequency movements, only worsening slightly when the movements becomes fastest. For these two sets, when movements are the fastest, the largest errors appear to occur at the boundaries of the sinusoidal movement (which has interesting consequences to speed/accuracy trade offs). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 5.5 Comparing the average performance of ANNs designed to predict joint angles from one of four sensory sets when tendon stiffness parameters are varied. Changes in motor damping can also be seen here and are denoted by line styles in the legend. It is interesting to note that the performance of ANNs trained with tendon tension information (left two columns) perform worse as tendons become more rigid, while the performance of ANNs trained with motor information only (right two columns) do not appear to be affected by changes in tendon stiffness. . . . . . . . . . . . . . 158 xxiii 5.6 Comparing the average performance of ANNs designed to predict joint angles from one of four sensory sets when motor damping is varied. Changes in tendon stiffness can also be seen here and are denoted by line styles in the legend. We again see from the vertical separation of the lines in the left two columns that ANNs trained with tendon tension information perform worse as tendons become more rigid—a trend not observed in the ANNs trained with motor information only (right two columns). Additionally, as seen in Figure 5.5, there appears to be no trends in performance with respect to motor damping except for those ANNs trained on all motor states (including acceleration, right column), which intuitively makes sense as higher damping may mean more useful information in the motor acceleration states. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 5.7 Heatmap representation of the relationship between tendon stiffness, motor damp- ing, and the performance of ANNs that utilize one of four sensory sets to predict joint angle from limited-experience babbling data. The logarithm of the perfor- mance (MAE) has been placed in each square where negative values correspond to good performance. Note that, as seen in Figures 5.5 & 5.6, the ANNs trained with tendon tension information (left two columns) have worse performance at higher tendon stiffnesses while the ANNs trained on motor information do not show such a trend. Additionally, we can see a slight correlation between motor damping and the performance of ANNs trained with all motor states (including acceleration, left columnn), but no identifiable trends for the other sets. . . . . . . . . . . . . . . . 160 5.8 Bar plots of the average performance for ANNs for each sensory set when the ten- don stiffness values are very high (See Section 5.3.3 for explanation). We find that ANNs trained with tendon tension information (the Bio-Inspired Set and the set of All Available States), still outperform those ANNs trained on motor infor- mation alone (the sets of Motor Position and Velocity Only and All Motor States). However, the difference is not nearly as large as in the previous experiments (3 orders of magnitude) because (i) the ANNs trained on motor information only im- proved their performance by nearly one order of magnitude and (ii) the ANNs that train with tendon tension information continue the trend of worsening performance when tendon stiffness increases (increasing errors by nearly 1 order of magnitude). 162 5.9 The training performance versus epoch number for 50 trials for each sensory set overlaid to show trends (left) and the average number of epochs used before ter- minating training (right). Compared to Figure 4.13, the training does not appear to be different for the ANNs trained with tendon tension information (with similar convergence rate, final performance, and number of epochs used), but the train- ing of ANNs that use the Motor Position and Velocity Only or All Motor States sets now have longer training periods (i.e., more epochs) and better performance. This is consistent with the notion that as the tendons become more rigid, the motor states becomes more useful and longer training periods are needed to extract that information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 xxiv 5.10 Polar bar plots to demonstrate the performance of ANNs trained on each sensory set when generalizing to different movements as a function of joint angle. Most of the performance values are consistent across the joint angle space with the ex- ception of the task where both the joint angle and joint stiffness were varied sinu- soidally (with joint stiffness at twice the frequency to have maximum stiffness at the boundaries of the movement). This would indicate that these ANNs are making better use of the tendon tension information at higher stiffnesses (and performing worse at lower stiffnesses where the nonlinearity of the tendon-tension deforma- tion curve becomes more disporportionate). . . . . . . . . . . . . . . . . . . . . . 164 5.11 Average performance heat maps for each sensory set and each generalization move- ment as a function of both joint angle and joint stiffness. As it can be seen, the ANNs trained on tendon tension data (left two columns) still outperform the ANNs trained only on motor information (right two columns) and appears to better gen- eralize at (i) lower stiffness values and (ii) joint angles that neither correspond to the vertical position or the boundaries of the range of motion. . . . . . . . . . . . 165 B.1 Approximation of MT geometry as a flattened parallel bundle of pennated muscle fascicles in series with tendon such that the (constant) internal tendon length is not necessarily parallel to the external tendon (askew by angle ). The muscle fascicles are still askew from the line of action of the MT by the pennation angle, . The projected length of the entire muscle “belly” is defined asL mb and is equal to the internal tendon and the muscle fascicle length (l m ) being projected back onto the line of action (Gans and Bock, 1965; Gans, 1982; Otten, 1988; Zajac, 1989). . 195 B.2 Contour plot depicting the value of the coefficient ofl 2 T;int in Eq. B.10a (sin( o + ) sin( o )), which captures the vector contribution of the internal segment of the tendon as the angle by which it is askew () changes with respect to muscle contraction. Note that this value is near zero when the angles do not change by very much. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 xxv Abstract Estimates of limb posture are critical for the control of robotic systems. This is generally accomplished by utilizing on-location joint angle encoders which may complicate the design, in- crease limb inertia, and add noise to the system. Conversely, some innovative or smaller robotic morphologies can benefit from non-collocated sensors when encoder size becomes prohibitively large or the joints are less accessible or subject to damage (e.g., distal joints of a robotic hand or foot sensors subject to repeated impact). These concerns are especially important for tendon- driven systems where motors (and their sensors) are not placed at the joints. Here we create a novel framework that can utilize limited-experience to provide accurate and efficient joint angle estima- tion during dynamic tasks using non-collocated actuator and tendon tension measurements. We draw inspiration from Nature where (i) muscles and tendons have mechanoreceptors, (ii) there are no dedicated joint-angle sensors, and (iii) dedicated neural networks may perform sensory fusion. To do so, we simulated an inverted pendulum driven by an agonist-antagonist pair of motors that pull on tendons with nonlinear elasticity, then compared the contributions of different sets of non-collocated sensory information (like motor positions or tendon tensions) when training artifi- cial neural networks (ANNs) to estimate joint angle. By comparing performance across different movement tasks we were able to determine how well each ANNs (trained on the different sensory sets of babbling data) generalizes to unlearned tasks (sinusoidal and point-to-point). We find that training an ANN with motor positions and velocities as well as tendon tension data produces more accurate estimates of joint angles than those ANNs trained without tendon tension data across all generalization movements. Additionally, we find that these results are robust to changes in mechanical plant parameters like tendon stiffness and motor damping and performs consistently well across different types of movements (from slow to ballistic). We conclude that regardless of the tendon architecture, the actuator behavior, or the movement complexity, tendon tension infor- mation is useful (if not vital) when estimating joint angles from non-collocated sensory signals. xxvi Additionally, this work—which shows that tendon tension information is invaluable for the esti- mation of joint angles by a simple neural network for a tendon-driven system—provides evidence to support the notion that in biological systems tendon tension information (conveyed by Golgi tendon organs) can be used with spindle afferents to better estimate limb posture. xxvii Introduction Learning to move is easy. Just ask my 1 year old son. Against all odd, armed only with untrained neural networks and limited experience, he has somehow solved the dreaded inverted- pendulum problem and started walking. He has resolved redundant strategies through numerous iterations of trial and error and emerged victorious with a “good enough” solution to the problem, much to his parents’ paradoxical relief and concern. And yet, learning how we move is anything but easy. Physiologists have long debated how the nervous system resolves muscle redundancy (i.e., having more muscles than joints), usually invoking some global or local optimality principle to make sense of the data (Scott, 2004; Loeb, 2012; de Rugy et al., 2012; Berret et al., 2019; Theodorou and Valero-Cuevas, 2010; Burdet et al., 2001; Todorov and Jordan, 2002). Meanwhile, roboticists have struggled to understand how to robustly control unstable, serial linkage systems when generalizing to different tasks/environments (Nicolescu and Matari´ c, 2003) or when asked to safely interact with humans (Zinn et al., 2004). The former can often be resolved by some model-based control strategy when sufficient informa- tion is available to adequately model the plant and its environment (Di Carlo et al., 2018) or by some machine learning approach where the input-output mapping is autonomously learned (Mar- janinejad et al., 2019a). The latter has been addressed by introducing compliance in the actuators to decrease the dangers of potentially large effective limb inertia (Hyun et al., 2010; Bicchi et al., 2008; Tonietti et al., 2005). Drawing inspiration from biological systems that do not appear to have these problems, one field is situated squarely in between these two research questions; a machine learning approach to understanding compliant tendon-driven systems. 1 0.1 Background 0.1.1 Why study compliant tendon-driven systems? Tendon-driven robots are becoming popular due to a number of advantages these designs can provide (Valero-Cuevas, 2016; Andrychowicz et al., 2019; Marjaninejad et al., 2019b). Elastic tendons can increase energy efficiency by storing potential energy and can protect actuators from impacts by dissipating energy upon impact (Laurin-Kovitz et al., 1991; Pratt and Williamson, 1995; Pratt, 2002; Mazumdar et al., 2017). Additionally, tendon routing options offer flexibility to how torques and angular velocities at the motors are converted to torques and angular velocities at the joints (Lee and Tsai, 1991; Kobayashi et al., 1998; Marjaninejad and Valero-Cuevas, 2019; Mar- janinejad et al., 2019a). Most importantly, tendon-driven systems offer flexible placement options for the actuators, which eliminate the need for motors to be placed on the joints themselves. Prox- imal actuator placement moves the center of mass towards the body of the robot thereby reducing limb inertia and allowing for more efficient displacement in quadrupeds or anthropomorphic robots (Jacobsen et al., 1986). For both biological and bio-inspired tendon-driven systems where antagonistic actuators pull on compliant tendons, control is often under-determined as the number of actuators exceeds the number of degrees of freedom (DOFs). Hidden in this redundancy problem lies another inherent benefit of tendon-driven systems masquerading as a obstacle; the ability to modulate joint stiffness by changing the amount of tension on the tendons. The ability to control both joint position and stiffness in these tendon-driven robots may be the key to creating compliant robots that can safely interact with humans and their environment, especially in the presence of uncertainty (Palli et al., 2008). 2 0.1.2 Statement of the Problem Most successful state-based robotic control strategies need to observe or approximate joint angles which is generally done by placing sensors on the joints (in the absence of alternatives such as visual feedback, Marjaninejad et al., 2019b,c). Although sensors in general have lighter mass than motors, this can still add unwanted inertia to the limbs. These on-location sensors are prone to motion noise and their wiring is often cumbersome and poses a potential risk of damage. These adverse effects become more pronounced for smaller, distal joints where the mechanical design may make the joints inaccessible (e.g., in the case of a tendon-driven finger in a robotic hand). One alternative solution, which biology seems to take advantage of, is to have non-collocated sensors (i.e., in the muscle and tendon instead of the joint) and use fusion of sensory information from actuators and tendons to predict joint angles. It is interesting to note that biological systems do not seem to have dedicated sensors that explicitly and uniquely encode joint angles. Instead, they have sensors for muscle (actuator) lengths and velocities (called muscle spindles; Crowe and Matthews, 1964) and for tendon tensions (called Golgi tendon organs; Appenteng and Prochazka, 1984) 1 . Previous work has emphasized that a functional (yet indirect) relationship exists between sen- sory states in general (e.g., muscle fascicle lengths and tendon tensions) and kinematic states (i.e., posture; Valero-Cuevas, 2016; Zajac, 1989; Hagen and Valero-Cuevas, 2017). It is therefore spec- ulated that these sensory signals may be integrated through their spinal and supraspinal projections to form internal representations of expected or virtual limb position (Scott and Loeb, 1994; Dim- itriou and Edin, 2008; Van Soest and Rozendaal, 2008; Kistemaker et al., 2013). The existence (and possible use) of this indirect relationship between sensory states and kinematic states in biology implies it may be possible to use sensory fusion in tendon-driven robots to infer joint angles from both actuator (e.g., motor angles) and structural (e.g., tendon tensions) sensors, thereby removing 1 There are additional biological sensors that detect stretch in the skin and synovial capsule, but these do not directly encode joint position either (Kandel and Schwartz, 2000). 3 the need for on-location joint angle encoders. While it is sometimes possible to derive analytical relationships among tendon tensions, mo- tor rotations, and limb posture given the precise equations for the kinematics and dynamics, in practice it is often impractical or impossible to obtain accurate and time-invariant models of such nonlinear dynamical systems (Bongard et al., 2006; Marjaninejad et al., 2019b). Furthermore, even if an accurate model of the system were available, these relationships (i) would not general- ize across changes in mechanical designs or tasks and (ii) will become increasingly inaccurate as the plant suffers mechanical changes due to either damage or normal wear and tear (Palli et al., 2012). Therefore, data-driven systems that can efficiently create mappings between sensory in- formation are preferred in practical applications (Bongard et al., 2006; Marjaninejad et al., 2018; Kwiatkowski and Lipson, 2019). To that end, the objective of this dissertation is to determine if it is possible to predict joint posture in a compliant tendon-driven system using non-collocated sensory information and artifi- cial neural networks (ANNs) trained on limited motor babbling experience. Limited experience is critical for the autonomous control of physical robotic systems because current machine learn- ing algorithms require large data sets that can lead to robot body wear and damage—and timely performance. Furthermore, we will explore whether and how the inclusion of tendon tension infor- mation improves the performance of such predictions. Lastly, we will discuss how changes in (i) movement requirements or (ii) mechanical properties of this system will affect the utility of such prediction algorithms. 0.2 Significance As previously stated, musculoskeletal control is under-determined, with redundancy in both the input (muscle redundancy) and the output (kinematic redundancy; Bernstein, 1967). Learning to overcome these redundancies to produce smooth, accurate movements is no small feat, yet children 4 (and all terrestrial mammals, for that matter) learn to walk and play relatively quickly compared to their robotic imitators (Spinka et al., 2001; Spelke and Kinzler, 2007; Adolph, 2008; Adolph et al., 2012). It is debatable whether muscle redundancy exists at all because real-world tasks have many more constraints than the simple laboratory tasks studied (Loeb, 2000; Valero-Cuevas, 2016). When physical tasks have sufficiently many constraints (like keeping joints together while regulat- ing limb kinematics, kinetics, impedance, and the sequencing of tasks), vertebrates have evolved to an ubiquitous stable phenotype with apparently “too many” muscles to satisfy that many con- straints while meeting the desired mechanical goal of the task. Regardless, neuroscientists and engineers take the reductionist approach in modeling where optimization is useful (Todorov and Jordan, 2002; Scott, 2004), even though there is also evidence that learning a “good enough” solu- tion through experience, repetition, and refinement is a more realistic biological strategy (de Rugy et al., 2012; Marjaninejad et al., 2019b). The better question is not whether the nervous system can find a solution (it obviously does), but how robust and reliable control is even possible in the context of inaccurate or non-existent internal models, noisy state estimators, and nonlinearities in sensing and actuation (Loeb, 2012). In an effort to understand how the nervous system can controls these nonlinear, redundant systems, researchers have employed inverse kinematic approaches that neglect tendon elasticity for conceptual and mathematical convenience (Stanev and Moustakas, 2019; Hagen and Valero- Cuevas, 2017; Berry et al., 2017; Millard et al., 2013; Scott, 2004; Jordan and Wolpert, 1999; Valero-Cuevas et al., 2009). While this approach is appealing because it reduces the model com- plexity, it (1) fails to capture the behavior of the muscle fascicles entirely as it ignores tendon deformation and (2) removes the intrinsic benefits of nonlinear, elastic tendons; e.g., postural sta- bility (Milner, 2002; Mussa-Ivaldi et al., 1985), joint stiffness modulation (Perreault et al., 2002, 2001; Osu and Gomi, 1999), potential energy storage (Alexander and Bennet-Clark, 1977; Alexan- der, 1988; Shadwick, 1990), and actuator protection from impact (Konow et al., 2012; Roberts and Azizi, 2010). Therefore, tendon must be included in computational models to accurately predict muscle fascicle mechanics and to take advantage of these dynamic features of tendon. Only then 5 can we put ourselves in the place of the nervous system to understand neuromuscular control. Several interesting and important consequences emerge when tendon elasticity is incorporated in musculoskeletal models. Assuming a simplified, parallelogram model of pennated muscle fas- cicles in series with elastic tendon—the functional musculotendon (MT) unit coined by Zajac (1989)— there exists a functional relationship between the kinematically-derived MT excursion, the change in muscle fascicle length, and the tension-specific tendon deformation. This relation- ship intuitively updates the actuator state by the resulting kinematic states and the (potentially la- tent) effects of the tendon dynamics. And while this equation is useful for updating fascicle length approximations, it has implications to both biological motor control and the control and design of tendon-driven, bio-inspired robots. Specifically, it is possible to utilize sensory information (either gathered from proprioceptors in biology or transducers in hardware) to predict the kinematics of a system, assuming that this functional relationship is (i) well defined (even if only temporarily) and (ii) possible to learn. Machine learning algorithms provide a promising platform on which to test whether and how it is possible to learn and exploit this relationship for estimating kinematic states with limited experience (Marjaninejad et al., 2019a). These algorithms are ideal for robotic applications where we wish to estimate posture from sensory data as they do not have to contend with poor model estimates of nonlinear systems (Gijsberts and Metta, 2013). For the physiological applications, these ANNs provide a parallel learning approach where we can speculate on whether the existence of such a functional relationship could be used. 0.3 Innovation The main contribution of this work is that it will provide valuable information regarding the integration of sensory information (whether that be biological or in hardware) regarding whether and how it can be combined to approximate kinematic states. We will systematically identify how 6 and when changes to the mechanical parameters that describe the dynamics of both sensory states and kinematic states break the coupling between actuator states and kinematic states. We will additionally discuss under what dynamic task requirements certain sensory information gains or loses usefulness for kinematic state estimation. We will make speculations regarding the neural control of movement and how the existence of proprioceptors that encode both tendon tensions and muscle mechanics may, in fact, be integrated to allow for virtual position estimation. This research will also provide a limited experience machine learning algorithm by which it is possible for a system to learn to predict virtual limb postures from motor angle encoders and tendon tension strain gauges in order to either (1) remove the need for joint encoders or (2) control the system via its virtual posture (similar to the equilibrium point hypothesis). 0.4 Thesis Outline The thesis will be broken down as into two main sections; (1) the exploration of tendon-driven control and the role that tendon elasticity plays in it and (2) the exploitation of this role to better estimate posture from ANNs trained with tendon tension information. First we will discuss how approximating muscle/actuator mechanics from joint kinematics does not capture the true behavior of a tendon-driven system nor the relationship between kinematic states and actuator states. To do this, we will first explore how kinematic redundancy inherently produces differences in MT behav- ior, even for similar movements. To better understand the relationship between muscle/actuators and elastic tendons, we will then explore the control of the simplest redundant tendon-driven sys- tem (“tug of war” on a pendulum) to illustrate how nonlinear tendon elasticity decouples kine- matic states and actuator states. We will then derive a more accurate calculation of muscle fascicle lengths that account for MT excursions (limb kinematics) and tendon deformation (kinetics) to explicitly explore the relationship between muscle (actuator) states, tendon tension, and posture. Exploiting the knowledge of the existence of such a functional relationship between these actu- 7 ators, tendon tensions, and posture, we then create and train ANNs to test if the addition of tendon tension as an observable state increases the performance when predicting posture in a compliant tendon driven system. To do this we will explore the use of limited experience, low-frequency, white-noise motor babbling to exploit the input space to produce useful training data for these ANNs. Next we will explore the relationship between learning rate and performance, and how it pertains to the discussion of an adequate choice for ANN architecture. And lastly, we will explore the affect that changes in (1) movement frequency, (2) motor damping, and (3) tendon stiffness have on the performance of these ANNs. 8 Chapter 1 A consequence of kinematic redundancy is that similar movements can elicitdifferent musculotendon states Muscle fascicle behavior can help explain motor control strategies, but is difficult to measure in vivo. Kinematics, on the other hand, are readily measurable. When assuming stiff tendons and constant or near-zero muscle fascicle pennation angles, muscle fascicle behavior can be conve- niently approximated from the kinematically-derived musculotendon (MT) behavior (i.e., the sum of tendon and pennated muscle fascicle lengths; Grieve, 1978; An et al., 1981; Kurse et al., 2012; Valero-Cuevas, 2016). This first approximation of muscle fascicle behavior has been used exten- sively to make speculations regarding neural control, but is severely limited in its assumptions. This approximation does, however, provide evidence that kinematic redundancy leads to similar movements exhibiting different MT behavior. Research presented in the chapter was published in the Journal of Biomechanics under the title Similar movements are associated with drastically different muscle contraction velocities (Hagen and Valero-Cuevas, 2017). 1.1 Abstract We investigated how kinematic redundancy interacts with the neurophysiological control mech- anisms required for smooth and accurate, rapid limb movements. Biomechanically speaking, mus- culotendon (MT) excursions are over-determined because the rotation of few joints determines the lengths and velocities of many MTs. But how different are the MT velocity profiles induced by equally valid hand trajectories? We used an 18-muscle sagittal-plane arm model to calcu- late 100,000 feasible shoulder, elbow, and wrist joint rotations that produced valid basketball free throws with different hand trajectories (with identical initial and final hand positions and veloci- 9 ties). As a kinematic approximation, we equated muscle fascicle velocities as MT velocities by as- suming stiff tendons and negligible muscle pennation. We found large differences in the eccentric and concentric fascicle velocity profiles across many trajectories; even among similar trajectories. These differences have important consequences to their neural control because each trajectory will require unique, time-sensitive reflex modulation strategies. As Sherrington mentioned a century ago, failure to appropriately silence the stretch reflex of any one eccentrically contracting muscle will disrupt movement. Thus, trajectories that produce faster or more variable eccentric contrac- tions will require more precise timing of reflex modulation across motoneuron pools; resulting in higher sensitivity to time delays, muscle mechanics, excitation/contraction dynamics, noise, er- rors and perturbations. By combining fundamental concepts of biomechanics and neuroscience, we propose that kinematic and muscle redundancy are, in fact, severely limited by the need to regulate reflex mechanisms in a task-specific and time-critical way. This in turn has important consequences to the learning and execution of accurate, smooth and repeatable movements—and to the rehabilitation of everyday limb movements in developmental and neurological conditions, and stroke. 1.2 Introduction The pursuit of, say, the perfect basketball free throw relies heavily on practice. Yet only those of us capable of consistently accurate throws can be paid millions of dollars as elite athletes. But why is it that practice or mimicry alone do not suffice to accomplish a professional level of accuracy and repeatability? In recent work, we re-emphasized that the neural control of limb movements is in fact over-determined, where the rotations of a few joints determine length changes in many musculotendons (MTs) (Valero-Cuevas, 2016; Valero-Cuevas et al., 2015). While some muscles that are shortening during the movement can, of course, become lax, those that are lengthening must all do so by an appropriate amount so not to alter the desired movement. 10 As pointed out by Sir Charles Sherrington (1913), movement can be disrupted if even one mus- cle undergoing eccentric contraction fails to silence its stretch reflex appropriately. Sherrington spoke of reflex inhibition being an important factor in the coordination of movement and posture, where the inhibitory process is no less capable of delicate quantitative adjustment than the excita- tory process (Sherrington, 1932). This idea was later refined by a cohort of scientists (for overviews see Loeb (1984) and Prochazka et al. (1985)) as the explicit and context-dependent regulation of the fusimotor, or , system to control muscle spindle sensitivity independently of-motoneuron drive. For a few decades now, it has been well accepted that the modulation of spinal reflexes, including the inhibition of stretch reflexes, is an intrinsic and necessary feature of the neural con- trol of force, posture, and movement; and often a neurophysiological mechanism responsible for pathological disruptions such as spasticity and clonus (Zehr and Stein, 1999; Hultborn, 2006; Hi- dler and Rymer, 1999; Sanger et al., 2010). But the question remains: how accurately must spinal reflexes be modulated in natural movement? Here we investigate the neuromechanical relationships between kinematic redundancy, MT velocity, and by extension, muscle fascicle contraction velocities—and explore its consequences to muscle afferentation. Specifically, given that MT excursions are over-determined, we approx- imate the different muscle fascicle velocity profiles induced by different, but equally valid, hand trajectories for a basketball free throw. This serves as the neuromechanical foundation to discuss how kinematic and muscle redundancy are, in fact, severely limited by the need to regulate reflex mechanisms in a task-specific and time-critical way. We conclude by discussing how these funda- mental neuromechanical concepts have important neurophysiological consequences to the learning and execution of accurate, smooth, and repeatable athletic movements—and to the disruption and rehabilitation of everyday movements in neurological conditions and stroke. 11 1.3 Material and Methods The goal of this study was to determine whether and how different movement trajectories that meet the initial and final conditions for a successful free throw produce differences in muscle fascicle contraction velocities. 3cm 3cm Wrist F/E ECRB EDM ECRL EDC FCU APL PL FDS FCR ECU 3cm 3cm Elbow F/E TRIC EDC EDM ECU ECRB PRO FCR FCU FDS ECRL PL BRA BIC BRD FLEXION EXTENSION 3cm 3cm TRI DELTp BIC DELTa CB Shoulder F/E Figure 1.1: Schematic of our 18 muscle, 3 degree of freedom (DOF) model which considers sagittal-plane shoulder flexion/extension (F/E), elbow flexion/extension and wrist flexion/exten- sion only. Forearm pronation/supination and radial-ulnar deviation were excluded from our model as studies have shown player’s to typically keep the shoulder, elbow, wrist, and ball in the same plane as the basket (Knudson, 1993). For an overview of the muscles used and their posture- dependent moment arm functions see Valero-Cuevas (2016), Ramsay et al. (2009), and Holzbaur et al. (2005). 1.3.1 Arm kinematic model We used an 18-muscle sagittal-plane arm model with 3 DOFs— shoulder, elbow, and wrist joints—to calculate a family of 100,000 valid basketball free throws with different hand trajec- tories (using identical initial and final hand positions and velocities). The free throw motion is approximated by a planar 3-DOF model. While studies have shown that basketball players typ- ically keep the ball, wrist, elbow, and shoulder in plane with the basket (Knudson, 1993), this 12 purely kinematic model does not include the neuromuscular control needed to produce such planar movement. Anthropometry was used to estimate physiologically-reasonable upper-limb segment lengths for a hypothetical 183 cm tall (6 ft) basketball player throwing from the free throw line (Winter, 2009) and the forward kinematic (i.e., geometric) model of the arm was defined as ~ x = ~ G( ~ ) = 0 B B B B @ l 1 cos( 1 ) +l 2 cos( 1 + 2 ) +l 3 cos( 1 + 2 3 ) l 1 sin( 1 )l 2 sin( 1 + 2 )l 3 sin( 1 + 2 3 ) 1 + 2 3 1 C C C C A (1.1) ~ x = 0 B B B B @ G x ( ~ ) G y ( ~ ) G ( ~ ) 1 C C C C A (1.2) where the vector ~ = ( 1 ; 2 ; 3 ) T contains the three arm joint angles for sagittal-plane shoulder rotation ( 1 ), elbow flexion/extension ( 2 ), and wrist flexion/extension ( 3 ) 2 . The upper-arm, fore- arm, and hand segments are denoted by link lengthsl 1 ,l 2 , andl 3 , respectively. This full geometric model of the limb in the sagittal plane specifies the horizontal (G x ) and vertical (G y ) position of the endpoint of the hand with respect to the shoulder joint, as well as the sum of all angles relative to rest (G ; Valero-Cuevas, 2016). This is because a rigid body (i.e., the hand) has three DOFs on the sagittal plane: two positions,G x andG y , and one orientation,G . 1.3.2 Defining initial and final arm postures, hand positions and velocities Initial and final arm postures for the model, ~ i and ~ f , were chosen from average measurements of three sample free throw motions. Passing these joint angles through Eq. 1.1 generate initial and 2 Due to the full pronation of the forearm, the sign convention for wrist flexion/extension has been flipped in order to consistently associate positive angular velocity with flexion throughout the model. 13 (x f , y f ) Basket Height Player’s Height 0 0 Distance to Center of Basket ∆y ∆x α v f Player’s Height Ball Radius Basket Parameters Joint Angles at Release ∆x, ∆y v f = (v x , v y ) x f , y f α c b a Ball Radius α Figure 1.2: Calculation of the final hand endpoint velocity vector (~ v f ) necessary for a successful free throw. As this throw is well modeled as a ballistic problem, the independent variables are the angle of release () and the distance of the ball’s center from the basket’s center (x, y). (a) Utilizing the anthropometric geometric model described in Eq. 1.1, a player’s height and the joint angles at the point of release are utilized to find the position of the hand endpoint relative to the shoulder joint (designated here as the origin). (b) The radius of the ball and the parameters of the basket (i.e. height of the basket and horizontal distance of the shoulder joint to the basket’s center) are incorporated to find the necessary displacement parameters (x, y). (c) Utilizing these displacements and the angle of release (), ballistics equations are rearranged and used to solve for the necessary release velocity vector for a successful free throw. 14 final hand endpoint positions, ~ x i and ~ x f . As free throws begin from rest, the initial endpoint velocity is zero. Free throws are simple ballistic shots, and therefore the final endpoint velocity, ~ v f = (v x ;v y ) T , was set to the release velocity vector of the basketball necessary for a successful throw (See Figure 1.2 for detailed formulation). Illustrated in Figure 1.3(a), the initial and final arm positions and velocities are therefore defined as: ~ x i = (x i ; y i ; i ) T = (G x ( ~ i ); G y ( ~ i ); G ( ~ i )) T (1.3) _ ~ x i = ( _ x i ; _ y i ; _ i ) T = (0; 0; 0) T (1.4) ~ x f = (x f ; y f ; f ) T = (G x ( ~ f ); G y ( ~ f ); G ( ~ f )) T (1.5) _ ~ x f = ( _ x f ; _ y f ; _ f ) T = (v x ; v y ; 0) T (1.6) Note the final velocity of the hand as a rigid body, _ ~ x f , is a twist that contains the linear velocity of the hand endpoint as the release velocity for the basketball, accompanied by zero net angular velocity (Valero-Cuevas, 2016). Initial and final joint angular velocities ( _ ~ i and _ ~ f ) were calculated from their relationship with endpoint twists and the Jacobian of the limb evaluated at each respective posture (Eqs. 1.7 - 1.9; Valero-Cuevas, 2016), and is illustrated in Figure 1.3(b). J( ~ ) = 0 B B B B @ @Gx( ~ ) @ 1 @Gx( ~ ) @ 2 @Gx( ~ ) @ 3 @Gy ( ~ ) @ 1 @Gy ( ~ ) @ 2 @Gy ( ~ ) @ 3 @G( ~ ) @ 1 @G( ~ ) @ 2 @G( ~ ) @ 3 1 C C C C A (1.7) _ ~ x =J( ~ ) _ ~ (1.8) _ ~ =J 1 ( ~ ) _ ~ x (1.9) 15 (f) Hand Trajectory x(t)=G( θ(t)) Forward Kinematics (e) Time-history of joint angles in configuration space 0º 180º 90º 180º -90º 0º θ 2 θ 3 θ 1 Initial Final Clamped Cubic Spline (d) Generate time-history of joint angles using clamped cubic splines Random Angle (c) Set random break point for cubic spline Max Min Final Initial Final Initial Final Initial -90° 0° θ 3 Max Min 80° 135° θ 2 Max Min -10° 140° θ 1 θ f , θ f . θ i , θ i . θ=J -1 ( θ)x . . (b) Calculate Initial and Final joint angles from measurements and angular velocities from (a) Measure arm postures, build kinematic model, and calculate Initial and Final endpoint velocities (x f , x f ) . θ 3, f θ 2, f θ 1, f + + + (x i , x i ) . θ 3,i θ 2,i θ 1,i + + + Final (Release) Initial (Start) Initial Final 16 Figure 1.3: Overview of trajectory generation technique. (a) Initial and final arm postures (taken from averaged, sample measurements) were passed through the kinematic model to find the initial and final hand endpoint positions (~ x i and~ x f ) and twists ( _ ~ x i and _ ~ x f ) (See Figure 1.2 for overview of ballistics). (b) Initial and final angular velocities ( _ ~ i and _ ~ f ) were calculated from their relationship to endpoint twists by the inverse Jacobian matrix (J 1 ( ~ )) evaluated at their respective joint angles ( ~ i and ~ f ). (c) A random break point (i.e. knot or seed) was generated for each joint angle by uniformly sampling from the joint’s range of motion as well as from the time between initial and final postures (0-550 ms). (d) Time histories for each joint angle were generated using clamped cubic splines to create smooth, piece-wise polynomial trajectories with proper initial and final con- ditions. Steps (c)-(d) were repeated if a resulting trajectory exceeded the joint’s range of motion or if undesirable (and unrealistic) rotations or velocities were encountered. (e) Combining joint angle time histories resulted in angle-angle-angle trajectories in configuration space. (f) Passing these combined angle time histories through the geometric model (i.e. the forward kinematic model) generated a hand endpoint trajectory with appropriate initial and final positions and velocities. 1.3.3 Generation of multiple valid hand trajectories We used clamped cubic splines to generate the time histories of individual joint angles which, when combined, produced valid hand trajectories, allowing us to fix the appropriate initial and final hand positions and velocities across trials while exploring multiple valid trajectories between them. This process is discussed and illustrated in Figure 1.3(c)-(f). In brief, to produce a joint angle trajectory, a random break point (i.e., knot) was found by uniformly sampling the joints range of motion and the time between initial and final postures (assumed to have a conservative 550 ms movement duration chosen from observation; Figure 1.3(c)). These three points (initial, knot, and final) were connected by two piece-wise cubic polynomials that produced a smooth, continuous trajectory with proper initial and final conditions (See Figure 1.3(d)). A trajectory was rejected if, at any point, it exceeded the joint’s defined range of motion. Additionally, shoulder or elbow angle trajectories were rejected if they produced initially negative angular velocities as this initial joint extension is not typically seen (i.e., players initially flex both the shoulder and elbow to bring the ball up from rest at the initial posture). Steps (c)-(d) in Figure 1.3 were repeated until these criteria were met for each joint angle. 17 These joint angle trajectories were then combined to produce a trajectory in configuration space (Figure 1.3(e)) and this time-history of joint angles can then be passed through Eq. 1.1 to generate the resulting hand trajectory (Figure 1.3(f)). Steps (c)-(f) were then repeated until 100,000 feasible joint rotations were properly generated for the shoulder, elbow, and wrist joints that produced equally many joint angle time-histories in the configuration space and realistic hand trajectories in the endpoint space. Figure 1.4 shows 20 such random trajectories in configuration space and the corresponding endpoint space trajectories. 0º -90º 135º 90º 135º 0º θ 1 θ 2 θ 3 Final Initial Initial Final (a) Sample Configuration Space Trajectories (b) Sample Endpoint Space Trajectories Figure 1.4: 20 trajectories in (a) configuration space and (b) endpoint space from a uniformly sampled solution space. These trajectories serve as examples of the solution space while individual trajectories will be explored further later in the analysis. 1.3.4 Approximation of normalized muscle fascicle velocities Existing literature was utilized to construct a posture-dependent moment arm matrix (R( ~ ) in Eq. 1.10; Valero-Cuevas, 2016; Ramsay et al., 2009; Holzbaur et al., 2005). The moment arms of these muscles at any particular joint configuration ( ~ ) can be used to estimate the changes in MT excursion (~ s) associated with changes in joint angles ( ~ , Eq. 1.11, An et al., 1983; Valero- 18 Cuevas, 2016). Assuming stiff tendons, we can estimate muscle fascicle velocities for each of the 18 muscles included in our model (~ v m ) as the MT excursion time derivative ( _ ~ s, Eq. 1.12). For ease of comparison across muscles, muscle fascicle velocities were normalized by their respective optimal muscle fiber lengths (Zajac, 1989). Figure 1.5 illustrates the normalized muscle fascicle velocity profile for a random trial. R( ~ ) = 2 6 6 6 6 4 r 1;1 ( ~ ) r 1;2 ( ~ ) r 1;18 ( ~ ) r 2;1 ( ~ ) r 2;2 ( ~ ) r 2;18 ( ~ ) r 3;1 ( ~ ) r 3;2 ( ~ ) r 3;18 ( ~ ) 3 7 7 7 7 5 (1.10) ~ sR( ~ ) T ~ (1.11) ~ v m ~ v MT = d~ s dt R( ~ ) T d ~ dt (1.12) ~ v m R( ~ ) T _ ~ (1.13) It is important to note that Eq. 1.13 represents an over-determined system of equations because ~ v m 2R 18 (an 18-dimensional vector), _ ~ 2R 3 (a three-dimensional vector), andR( ~ ) T 2R 18 3 (18 equations with three variables). Therefore, the values of three angular velocities influence the values of 18 muscle fascicle velocities. Given that a limb movement is a sequence of joint angular velocities, a given limb movement is only possible if all muscles are able to adopt appro- priate muscle fascicle velocities as the movement progresses. For muscles undergoing concentric (i.e., shortening) contractions, it is possible for some of them to become lax and still allow the limb movement. But if any muscle that is lengthening (through either eccentric contraction or passive stretch) fails to do so at any time during the movements (e.g., because of failure or in- 19 “Upstroke” “Power Stroke” Initial Final Concentric Eccentric 0 5 l m,o / s -5 l m,o / s Figure 1.5: Normalized muscle fascicle velocities vs. time for a randomly selected free throw attempt both in its entirety (left) and bounded by5 optimal muscle fascicle lengths per second (l m;o =s) (right) to highlight the upstroke phase. Muscles that only cross the shoulder are shown in navy, while the biceps and triceps muscles are shown in orange. The remaining muscles that cross the elbow and/or wrist are shown in light blue. Note that there exist two major zeros crossings during which many of the muscles change the direction of their contractions. The first instance occurs during the upstroke where the wrist extends as elbow flexes with similar but opposite angu- lar velocities–while the shoulder continues to rotate upwards causing the net MT excursion time derivative for the bi-articulating muscles of the elbow and wrist to offset each other. The second instance defines the start of the power stroke phase where either the angular velocities of the elbow and/or wrist create a net zero MT excursion time derivative or the muscles of the shoulder change the direction of their contractions (if they do at all). The muscles that only cross the shoulder do not exhibit this first zero crossing as they typically experience contraction throughout the entire upstroke without a change in direction (i.e. constant upward rotation about the shoulder joint), but will often exhibit the second zero crossing as a result of the cubic spline algorithm and the local extrema generated in the shoulder angle trajectory. As the biceps and triceps muscles cross over both the shoulder and elbow joints, they do not experience a zero crossing during the upstroke phase (i.e. the direction of both rotations are consistent and nonzero during this phase), but they do, however, exhibit typical zero crossings at the start of power stroke phase brought on by a major change in the direction of elbow rotation. ability to regulate or silence its stretch reflex; Loeb, 1984; Prochazka et al., 1985), the movement will be disrupted (Sherrington, 1932; Valero-Cuevas, 2016). The distinction between the under- determined nature of the control of joint torques (with many solutions) vs. the over-determined nature of the control of joint rotations (with at most one solution) is often lost in the biomechan- ics and neural control literature (Valero-Cuevas, 2016). When controlling individual afferented muscles, the neural control of limb force is fundamentally different from that of limb movement. 20 Not only are the governing equations for force and motion control different (Yoshikawa, 1990; Valero-Cuevas, 2016), but the former requires combining muscle forces to produce specific net joint torques, whereas the latter requires coordinating muscle forces while regulating reflexes to allow eccentric/concentric contractions compatible with the desired joint angles and angular ve- locities (Sherrington, 1913; Loeb, 1984; Prochazka et al., 1985; Mah and Mussa-Ivaldi, 2003; Venkadesan and Valero-Cuevas, 2008; Valero-Cuevas, 2016). 1.3.5 Definition of movement phases We considered each throw to have three phases: the upstroke, the power stroke, and the follow- through (Figure 1.5). The upstroke is the initial phase of the throw where the ball is brought from rest at the initial posture upwards towards a cocked position (i.e., when the velocities of the muscles that cross the elbow and/or wrist changed from eccentric to concentric contraction, or vice versa). As most muscles will undergo this change in the direction of their contraction velocity at different times (as dictated by their unique relationships between joint angles and moment arm values described in Eqs. 1.10-1.13), the upstroke is uniquely defined for each muscle during each trajectory. In the rare cases where none of the muscles of the shoulder exhibits this change in sign of contraction velocities, the upstroke is defined as the initial 520 ms of the motion. The power stroke is defined as the phase of the throw immediately following the upstroke. And lastly, the follow-through starts once the final endpoint velocity has been achieved. Here the wrist flexes as the ball is released. As stated previously, we do not consider this last phase as we only consider purely ballistic throws, and the mechanics of ball release and possible backspin are tangential to this study. Figure 1.5, therefore, only shows the two main phases of interest for our study. 21 1.3.6 Calculation of eccentric and concentric contraction velocity costs Each trajectory was assigned cost values for eccentric and concentric contraction velocity mag- nitudes separately. Given that all trajectories had similarly large contraction velocities during the power stroke phase, we categorized trajectories as per their contraction velocities during the up- stroke phase only. These cost values were defined as the Euclidean norms of the maximal eccentric (Eq. 1.14) and concentric (Eq. 1.15) contraction velocities for each muscles during that phase: Eccentric Cost = v u u t 18 X i=1 max(jv m;i j) 2 (forv m;i 0) (1.14) Concentric Cost = v u u t 18 X i=1 max(jv m;i j) 2 (forv m;i 0) (1.15) 1.4 Results Using the computational method described above, we generated 100,000 realistic throwing mo- tions that can achieve successful free throw hand trajectories. Surprisingly, we found that although the trajectories all clustered around a stereotypical path (Figure 1.4), there was a large distribution in both eccentric and concentric contraction velocity costs across them. Figure 1.6 shows both the histograms for eccentric and concentric costs separately, and a heat map of their joint distribution. This demonstrates that there exists a wide range of trajectories with very different eccentric and concentric costs that are capable of accomplishing the final hand position velocity for a valid throw. Furthermore, when sampling from these valid trajectories, we find cases where even kine- matically similar basketball free throws can have different contraction velocity costs (see sample trajectories 1, 2, and 3 in Figures 1.7). That is, the trajectory of the endpoint of the hand can be very similar across these three sample trajectories, but their paths in configuration space can be very different, as are their eccentric/concentric muscle fascicle velocity costs which can differ by 22 3% 2% 10 0 10 1 10 2 10 3 Number Of Valid Shots Frequency 0 Eccentric Cost Concentric Cost 0 2 2 4 4 6 6 8 8 10 10 (in l m,o / s) (in l m,o / s) Figure 1.6: Two dimensional histogram of the eccentric and concentric costs of all trajectories as defined by equations 1.14 and 1.15, respectively (in normalized unitsl m;o =s (Zajac, 1989)) with one dimensional histogram axes overlays. a factor of 3. Conversely, we find that trajectories can also vary in endpoint space while incurring similar costs. Figure 1.8 demonstrates this observation by showing the 20 most similar trajectories to our sample trajectories in terms of cost values. As seen in the bottom three rows of Figure 1.9, these three similar sample trajectories follow similar trends, but also exhibit subtle differences in individual angular trajectories (see stick figures at top and angle plots at bottom). Specifically, shoulder angle trajectories vary little in ranges or slopes, elbow angle trajectories have similar ranges with different slopes, and wrist angle trajecto- ries have increasingly larger ranges and slope magnitudes (cf. across columns). Given that muscle fascicle velocities are related to joint angular velocities, changes in the slopes of joint angles across trajectories have large effects on contraction velocities as seen in Figure 1.9. Across these three sample trajectories, angle slopes vary little during the first third of the upstroke phase and, as a result, the magnitudes of muscle contractions are similar during that early period. Increasing slope values for both the elbow and wrist angles appear to heavily influence the magnitudes of contrac- tion velocities, as seen by the remainder of the upstroke phase for sample trajectories 1, 2, and 3. 23 0º 180º 90º 180º -90º 0º Elbow Angle Wrist Angle Shoulder Angle Initial Final 1 2 3 0 0 2 2 4 4 6 6 10 Concentric Cost Eccentric Cost 10 8 8 2 1 3 10 0 10 1 10 2 10 3 Number Of Valid Shots (in l m,o / s) (in l m,o / s) Final (Release) 10 cm Initial (Start) 3 1 2 Figure 1.7: Two dimensional histogram of the eccentric and concentric costs of all trajectories as defined by equations 1.14 and 1.15, respectively (in normalized unitsl m;o =s) (bottom right) with sample trajectories 1, 2 and 3 plotted in configuration space (top right) and endpoint space (left). 10 cm Figure 1.8: Endpoint space plots of sample trajectories 1, 2, and 3, overlaid with their 20 most similar trajectories, respectively, as per eccentric and concentric cost values (gray). 24 We quantified how similar the 100,000 trajectories were by measuring how well they followed a given valid path. We took the sample trajectory 1 shown in Figures 1.7-1.9 as the reference trajectory, and calculated the average residual per time step (i.e., Euclidean distance in the sagittal plane) between it and each of the remaining 99,999 trajectories. The frequency and cumulative histograms of these residuals are shown in Figure 1.10(a). We find that 50% of them have mean residuals 14.60 cm, with a median of 11 cm. By comparison, the sample trajectories 2 and 3 had mean residuals of 2.18 cm and 5.14 cm, corresponding to the 0.165% and 4.465% cumulative percentiles, respectively. Interestingly, even though sample trajectories 2 and 3 are among the 5% most similar to sample trajectory 1, their eccentric and concentric velocity costs differ greatly as show in Figure 1.7. In fact, even the 100 most similar trajectories to sample trajectory 1 (with average residual error per time step values 1.96 cm) have a fairly wide distribution in both eccentric and concentric costs with nearly four- and two-fold increases in cost ranges, respectively, as seen in Figure 1.10(b)-(c). 1.5 Discussion It is, of course, undeniable that there are many ways in which one can coordinate joint rotations to smoothly and accurately produce a given trajectory of the endpoint of a limb (in this case, the hand during a basketball free throw). A first important result from this work is that the time histories of muscle fascicle lengths and velocities are not necessarily obvious for the multi-joint, multi-muscle limb—and that a given muscle can exhibit both eccentric and concentric contractions during a smooth and continuous hand trajectory such as the basketball free throw. The MT length changes and velocities are, in fact, specified by the over-determined multi-dimensional matrix- vector Eqs. 1.10 - 1.13 that are a function of joint angles and angular velocities, moments arms, and link lengths. By assuming stiff tendons, we form an over-determined relationship between joint angular velocities and muscle fascicle velocities; shown graphically in Figures 1.4, 1.5, 1.7, and 1.9. 25 We selected a few hand trajectories to explore, in detail, the relationships among joint angles and angular velocities, muscle fascicle velocity profiles, and hand trajectories. In particular, we 10 cm Final Initial Elbow Angle Shoulder Angle Wrist Angle 0 -10° 140° 80° 135° -90° 0° 1 2 3 Eccentric Contraction Concentric Contraction +5 l m,o / s -5 l m,o / s Figure 1.9: Sample trajectories 1, 2, and 3 shown in endpoint space (top) with their corresponding normalized muscle fascicle velocity profiles and joint angle trajectories (bottom 3 rows). Note that the dotted lines in the velocity plots indicate5l m;o =s, while dotted lines in the joint angle plots indicate the allowable range of motion for each joint. 26 50% 14.60 0% 100% Cumulative Frequency 6% 3% 0% Frequency 0 30 60 Average Residual (cm) 10 0 10 1 Number Of Valid Shots Frequency 0 0 2 2 4 4 6 6 8 8 10 10 30% Eccentric Cost Concentric Cost 90% (in l m,o / s) (in l m,o / s) Histogram of average residual per time step when comparing sample trajectory 1 to all other trajectories with cumulative distribution overlaid (a) Endpoint space plots of 100 most similar trajectories to sample trajectory 1 (avg. residual ≤ 1.96 cm) (b) Joint distributions for 100 most similar trajectories to sample trajectory 1 (c) Figure 1.10: (a) Histogram of the average residual per time step generated when comparing sample trajectory 1 to all remaining trajectories with cumulative distribution (solid black line) overlaid. Half of the trajectories had average residual values 14.60 cm while the mode of this distribution was 11 cm. (b) Endpoint space plot of the 100 most similar trajectories compared to sample trajectory 1 (green) as determined by the average residual per time step ( 1.96 cm). (c) The individual distributions (axes overlays) and the joint distribution of these most similar trajectories (in normalized unitsl m;o =s). selected sample hand trajectories 1, 2, and 3 (Figures 1.7, 1.8, 1.9, and 1.10) to make the point that there exist hand trajectories that are similar to each other, yet may have very different muscle contraction velocities (Figure 1.7). Conversely, having similar muscle contraction velocities does not imply that the hand trajectories will be similar (Figure 1.8). The former point is explored further in Figure 1.10 by using sample trajectory 1 as a reference. Figure 1.10(a) shows how sample trajectory 1 is not an outlier because the median of the residual difference to all other 99,999 trajectories is lower than the mean of that difference. We then show the 100 most similar trajectories to it (Figure 1.10(b)) exhibit a large spread in muscle contraction velocities. While we could repeat this analysis with other trajectories as a reference, Figures 1.7-1.10 suffice as clear counterexamples to the notion that similarity in hand trajectory implies similarity in muscle contraction velocities, and vice versa. To produce a given movement trajectory, all muscle activations must be appropriately coordi- nated, and afferent feedback from lengthening muscles must be appropriately tuned. Given muscle redundancy to produce joint torques, the efferent (outgoing) motor signals producing a limb move- 27 ment may vary without changing the limb trajectory. But there is no such redundant relationship between limb kinematics and afferent (incoming) sensory signals. As per the over-determined nature of Eq. 1.13, a given limb movement fully determines the time history of muscle fascicle lengths and velocities and, therefore, the muscle proprioceptive signals that affect stretch reflexes. Thus, any muscle that fails to appropriately lengthen (either by failure to modulate or silence the stretch reflex, or by inappropriate activation) will disrupt the movement trajectory in some way (Sherrington, 1913; Loeb, 1984; Prochazka et al., 1985; Valero-Cuevas, 2016; Valero-Cuevas et al., 2015). Some muscles undergoing concentric contractions could, in principle, go slack so long as other muscles contribute to drive the limb (Valero-Cuevas et al., 2015). But the timely and appropriate modulation of force and afferent sensitivity in muscles which are lengthening is critically necessary. Therefore, each movement trajectory will require distinct and time-sensitive neural strategies. Here we present a detailed analysis of kinematic redundancy in light of its con- sequences on muscle fascicle lengths and velocities, and by inference, on neural control. First, we reiterate that there are different valid hand trajectories (Figure 1.4) that produce a suc- cessful basketball free throw. What is less intuitive, and lacked characterization in prior literature, is that each valid trajectory can exhibit quite different muscle fascicle velocity profiles—which has unavoidable physiological consequences to muscle mechanics and muscle afferentation. Muscle afferentation is the physiological term used to describe proprioception that is specific to muscles. This involves the afferent (i.e., from the periphery inwards) sensory flow from muscle mechanore- ceptors to spinal, subcortical and cortical circuitry, as discussed further below. Our reasonable quantification of these consequences via eccentric and concentric costs (Eqs. 1.14 and 1.15) shows that valid trajectories are widely distributed and can differ by up to an order of magnitude in this cost landscape (Figure 1.6). This result alone suffices to revisit our understanding of kinematic redundancy. Kinematic re- dundancy is considered a learning and decision-making challenge for the nervous system, which must select a particular time history of joint angles and angular velocities from among the many 28 possible ones. At first, Bernstein proposed that the problem of kinematic redundancy is solved via a 3-stage approach, where initially some kinematic DOFs are ’frozen’ and then gradually re- leased as the nervous system learns to control all DOFs (Bernstein, 1967). More recently others have appropriately pointed out that kinematic redundancy must be evaluated with respect to the dimensionality of the task itself, and not just the number of kinematic DOFs of the limb (e.g., Newell and Vaillancourt, 2001; Ko et al., 2003). Along similar lines, others propose that, given the dimensionality of a desired task, kinematic redundancy allows for ’task-irrelevant’ joint angle changes and therefore gives rise to a nullspace or uncontrolled manifold for the task (e.g., Scholz and Sch¨ oner, 1999; Li et al., 2004). Moreover, the over-determined nature of muscle fascicle lengths and velocities has been implicated in the dimensionality reduction often seen in the control of limb function (Kutch and Valero-Cuevas, 2012; Brock and Valero-Cuevas, 2016). Along more computational lines, others propose that the decision-making challenge inherent to kinematic re- dundancy is addressed by the nervous system as an optimization problem (e.g., Loeb et al., 1990; Todorov and Jordan, 2002)—where the fitness of each possible movement solution is evaluated via a user-specified (and often debatable (Prilutsky, 2000)) cost function—e.g., energy expenditure (Crowninshield and Brand, 1981), jerk (Flash and Hogan, 1985), speed-accuracy tradeoff (Fitts, 1954). Our results, however, reveal an important Sherringtonian feature of kinematic redundancy by showing that valid trajectories are not intrinsically equivalent. Rather, they are intrinsically dis- tinct in their muscle fascicle velocity profiles. This has physiologiocal consequences to muscle mechanics and afferentation, and therefore require distinct and time-sensitive neural reflex mod- ulation strategies for their accurate, smooth, and repeatable execution (Sherrington, 1913, 1932; Loeb, 1984; Prochazka et al., 1985)—independently of any additional metabolic, state-dependent, or task-related cost function(s) the user may prefer to consider. In fact, we find that even very similar trajectories can have different muscle fascicle velocity profiles (Figures 1.7 and 1.9) 3 . 3 Strictly speaking, the endpoint trajectories of the hand are very similar while their hand orientations, which are not plotted, may differ given that the hand as a rigid body has three DOFs in the sagittal plane. Nevertheless, by construction, all trajectories strictly meet the initial and final conditions for hand position and orientation to produce a 29 Where does the need for distinct and time-sensitive neural strategies come from? Three papers (Loeb, 1984; Prochazka et al., 1985; Duchateau and Enoka, 2016) review the nature of muscle af- ferentation and its role during functional tasks. Briefly, separate neural commands control the bulk of the muscle (i.e.,-motoneurons), muscle spindle gains (i.e., -motoneurons), and stretch reflex pathways gains. The stretch reflexes resist muscle fascicle lengthening in a velocity-dependent way. Given that a movement trajectory determines the speeds at which muscles must lengthen, stretch reflexes must be modulated or silenced to allow such lengthening. Failure to modulate or silence the stretch reflex will prevent the desired joint rotations—and thus disrupt the move- ment trajectory unless other joint rotations compensate the disruption. Thus, different muscle eccentric contraction velocities (i.e., movement trajectories) will require distinct reflex modulation strategies; and faster muscle fascicle velocities will require appropriately faster and more time- critical reflex modulation strategies (Valero-Cuevas, 2016; Valero-Cuevas et al., 2015; Duchateau and Enoka, 2016). The different concentric contraction velocities may also require distinct and time-sensitive strategies to ensure that muscles do not go slack and cease to produce force, and to ensure that the work loops of all muscles contribute appropriately to accomplish the task well (Biewener and Daley, 2007). The distinct muscle fascicle velocity profiles for each valid trajectory reinforce the need for strict spatio-temporal constraints on the time-history of muscle coordination (R´ acz and Valero-Cuevas, 2013; Dingwell et al., 2010). Thus, trajectories that produce faster or more variable contraction velocities (i.e., higher costs, Figure 1.6) will likely require more precise timing of reflex modulation across motoneuron pools; resulting in higher sensitivity to time delays, force-length and force-velocity properties, short-range stiffness, excitation/contraction dynamics, noise, errors, perturbations, etc. This work justifies and enables future research directions by combining the Sherringtonian perspective with experimental (An et al., 1983) and analytical (Valero-Cuevas et al., 2015; Valero- Cuevas, 2016) demonstrations of the over-determined nature of muscle contraction velocities. This neo-Sherringtonian perspective towards kinematic redundancy has profound implications to the successful basketball free throw, as is the case in Bernstein’s hammering example (Bernstein, 1967), see Figure 1.3. 30 learning, execution, and rehabilitation of natural movements. For example, it leads to testable hy- potheses of why learning to produce accurate, smooth, and repeatable movements takes immense amounts of practice, even in typically developing children (Adolph et al., 2012), why so few of us can become elite musicians or athletes (Gladwell, 2008), and why rehabilitation requires mass practice (Lohse et al., 2014). Future experiments and computational simulations can also begin to tie neural plasticity and learning rates to the specific characteristics of a smooth movement. For example, do humans favor trajectories producing lower and less variable contraction veloci- ties? By extension, such an approach can help us understand how disruption of reflex mechanisms could lead to pathological movements such as spasticity, tremor and clonus (Zehr and Stein, 1999; Hultborn, 2006; Hidler and Rymer, 1999; Laine et al., 2016) in neurological conditions including cerebral palsy, stroke, Parkinsons disease and spinal cord injury (Sanger et al., 2010; De Gooijer- Van De Groep et al., 2016; Phadke et al., 2016; Agapaki et al., 2016). Similarly, perhaps one can design arm movement trajectories that are more effective for rehabilitation because they require less time-critical modulation of stretch reflexes. We conclude that moving smoothly, repeatedly, and well is neither a redundant nor a forgiving problem. It requires confronting and overcoming the over-determined problem of appropriately regulating- and -motoneuron activity in a task-specific and time-critical way across all muscles in the context of the nonlinearities of neurons, muscles, proprioceptors, and limb mechanics. 31 Chapter 2 Tendon elasticity decouples kinematic states from actuator states 2.1 Abstract Biological tendons deform nonlinearly with tension (Ker, 1981; Zajac, 1989; Shadwick, 1990; Brown et al., 1996). Interestingly, when a musculoskeletal system is made to follow the same limb kinematics with different control signals, muscle fascicle and musculotendon (MT) behavior could deviate by different amounts. In these under-determined systems, in order for different tendon tensions to generate the same joint rotations they must (1) share a particular solution to satisfy the dynamics, and (2) have a unique, homogeneous solution that has no effect on the dynamics. As such, this homogeneous solution is said to lie in the nullspace of the joint dynamics. And even though tendon tensions that lie in the nullspace have no additional effect on the kinematic states, that does not mean that they lie in the nullspace of the actuator dynamics. To explore how different choices of tendon tensions affect the relationship between joint kinematics and muscle mechanics, we simulated a simple pendulum actuated by two Hill-type muscles that pull on tendons with nonlinear stiffness. By applying one input constraint from an integrator backstepping algorithm to ensure that the pendulum follows a reference trajectory and by systematically varying the second constraint, we effectively sweep the nullspace of the kinematics. This allows us to demonstrate that tension-specific tendon deformation functionally and nonlinearly decouples the relationship between MT and muscle behaviors. Additionally, we find that the choice of initial tension (i.e., the pretensioning) will affect the initial muscle fascicle length, which also contributes to the amount by which MT and muscle decouple because of the muscle’s force-length property. We conclude that it is not possible to accurately predict the behavior of muscle fascicles by limb kinematics. 32 2.2 Introduction The previous chapter’s experiment relied on the assumption that tendons are completely inex- tensible. This assumption allowed for the over-determined relationship between joint kinematics and musculotendon (MT) to be extended to predict muscle mechanics. And while these approx- imations allowed for valuable inferences to be made regarding the consequence of kinematic re- dundancy on MT behavior, it does not fully capture the relationship between muscle and tendon. Therefore, the next logical step would be to introduce compliant tendons into a MT model. By doing so, the purely prescriptive model described in Chapter 1 (i.e., one where muscle fascicle lengths and velocities are over-determined by the prescribed joint kinematics) becomes a fully dy- namical model that requires control strategies to resolve muscle redundancy (i.e., joint kinematics are now a under-determined function of tendon tensions). The simple inclusion of compliant ten- dons changes the problem completely— we must now consider redundancy in the control space or its nullspace. As an example, to better visualize the nullspace, consider the following under-determined equa- tion: x 1 +x 2 = 4 (2.1) There are an infinite number of ways to solve this equation, but suppose we choose to solve this equation by minimizing the Euclidean norm of ~ x yielding the particular solution ~ x P = (2; 2) T (Figure 2.1, red). But if we relax this restriction that the norm be minimized, we can find additional solutions like~ x = (1; 3). This general solution can be rewritten as the sum of the particular solution and the homogeneous solution (i.e.,~ x = ~ x P +~ x H = (2; 2) + (1; 1)). Notice that the left hand side of Eq. 2.1 evaluated at the homogeneous solution is zero, thus the addition of this solution to the particular solution still satisfies Eq. 2.1. This homogeneous solution (and any other that satisfy x 1 +x 2 = 0 for that matter) are said to lie in the nullspace of Eq. 2.1. Extending this example to more complex, tendon-driven systems, one way to resolve the re- 33 x 1 x 2 x 1 + x 2 = 4 N = {(x 1 ,x 2 ) | x 1 +x 2 =0} x P = (2,2) T x H = (-1,1) T N Figure 2.1: Example of the nullspace of an equation inR 2 and its relationship to a particular solu- tion (red) given by some cost function (e.g., minimizing the Euclidean norm) and the homogeneous solution (blue) which lies in the nullspace (N) of the equation (grey) dundancy in the control signal is by optimizing some cost function to find a particular (single) so- lution. Physiologists often utilize this optimal control strategy to find control inputs that performs the desired task while minimizing cost functions like total input energy (Scott, 2004; Todorov and Jordan, 2002). These optimization techniques are widely used and can provide interesting infor- mation about motor control, but the cost functions are often not complete enough to fully describe physiological movement (Loeb, 2012). The other issue with this approach is that we find at most one solution for every cost func- tion/optimization so comparing different input strategies for the same movement will require differ- ent cost functions. Alternatively, if we consider inputs that occupy different areas of the nullspace, we can compare the entire cost landscape for identical movements and observe what happens when movements are “sub-optimal”. These nullspace strategies are kinematically equivalent as they pro- 34 duce no additional net joint torques other than those needed for the desired joint task/movement (much like a game of “tug of war” where no movement occurs when both sides pull with the same force but will occur when one side pulls harder than the other). To do this, we will need to constrain the inputs to those that produce the desired movement and then sample the nullspace. Here we consider the simplest redundant tendon-driven system whereby a single degree of free- dom (DOF) pendulum is controlled by two Hill-type muscle-like actuators that pull on compliant tendons. By using an integrator backstepping control technique, we find a single constraint on the input when the pendulum angle is made to follow a reference trajectory (e.g., a sinusoidal task). This single constraint reduces the dimensionality of the input space from a plane to a line inR 2 . This line corresponds to the nullspace of the task and by systematically sampling along this line, we find different control strategies that produce the same joint movement so that we can compare how compliant tendons affect the relationship between muscle mechanics and MT behavior. 2.3 Material and Methods 2.3.1 Pendulum Dynamics To understand the affect that compliant tendons have on the relationship between MT and muscle behavior, we will explore the simplest redundant tendon-driven system; a single pendulum controlled by two muscle-like actuators that pull on compliant tendons (Figure 2.2). The dynamics that govern the pendulum angle were derived using an Euler-Lagrange equation (Eqs. 2.2–2.4). L =TV = 1 2 ML 2 CM _ 2 +MgL CM cos() (2.2a) (2.2b) 35 d dt @L @ _ @L @ =r 1 ()f T;1 r 2 ()f T;2 ML CM =MgL CM sin() +r 1 ()f T;1 r 2 ()f T;2 (2.3a) (2.3b) = g L CM sin() + 1 ML 2 CM r 1 ()f T;1 +r 2 ()f T;2 (2.4) Note thatM andL CM correspond to the mass and center of gravity of the pendulum (Table 2.1), g is the gravitational constant, and f T;i and r i are the tendon forces and the moment arms that each “muscle” acts on the pendulum with (8i2f1; 2g). These (and all subsequent) parameters were chosen to simulate a human forearm actuated solely by the antagonist pair, biceps brachii and triceps brachii. For simplicity, the many heads of these muscles were considered to act in parallel as a single unit, and as such the parameters of the individual heads (when available) were lumped together to represent only two muscles; an agonist and an antagonist. The moment arms for these MTs are functions of the pendulum angle and were approximated by the functions provided by Ramsay et al. (2009) for the biceps and triceps muscles (Figure 2.3). 2.3.2 Tendon Tension Dynamics We then derived differential equations for the tensions on the tendons by manipulating the tendon tension-deformation equation derived in Brown et al. (1996) (Eqs. 2.5–2.6). f T;i =f max;i c T k T ln exp l T;i =l T;o;i L T r k T + 1 (from Brown et al. (1996)) (2.5) Table 2.1: Pendulum parameters. Values were chosen to approximate the dynamics of a human forearm. Plant Parameters Pendulum mass,M (kg) 1.6 Center of Mass,L CM (m) 0.30 36 LCM Mg r θ k ^ ^ ^ θ f T,1 f T,2 r 2 r 1 Figure 2.2: Pendulum controlled by two Hill-type muscle-like actuators that pull on nonlinearly compliant tendons to produce tensions (f T;i ) that produce torques at the joint through their moment arms (r i ). _ f T;i = f max;i c T l T;o;i 0 @ exp h l T;i =l T;o;i L T r k T i exp h l T;i =l T;o;i L T r k T i + 1 1 A _ l T;i = f max;i c T l T;o;i 0 @ exp h l T;i =l T;o;i L T r k T i + 1 1 exp h l T;i =l T;o;i L T r k T i + 1 1 A _ l T;i = f max;i c T l T;o;i 0 @ 1 1 exp h l T;i =l T;o;i L T r k T i + 1 1 A _ l T;i = f max;i c T l T;o;i 1 exp f T;i f max;i c T k T ! v MT;i (; _ ) _ l m;i cos( i ) =K T;i (f T;i ) v MT;i (; _ ) _ l m;i cos( i ) (2.6a) (2.6b) (2.6c) (2.6d) (2.6e) where f max;i and l T;o;i represent the maximum isometric tetanic muscle force and the optimal tendon length, respectively, and the parameters c T , k T , and L T r are related to the slope, radius of curvature, and lateral shift of the normalized tendon tension-deformation relationship, respec- tively 4 . Even though an explicit function exists to express tendon tension as a function of tendon deformation (which can be rewritten as the difference between MT length and muscle fascicle length scaled by its pennation angle, i ), we benefit from the recovered differential equation in Eq. 4 For more information on the meaning of these parameters, the reader is pointed to the Methods section of Chapter 3 where they are explicitly derived or to Brown et al. (1996) where they were originally formulated. 37 Moment Arm (m) 0.04 0.02 0 -0.02 r 1 (Biceps) r 2 (Triceps) Elbow Angle (deg) 140º 120º 100º 80º 60º 40º 20º Figure 2.3: Posture dependent moment arm functions (r i ) for “muscle 1” (biceps) and “muscle 2” (triceps) provided by Ramsay et al. (2009). The elbow angle is measured from the anatomical position and shoulder angle is assumed to be constant. 2.6d because (1) it is computationally easier to calculate the velocity of a MT instead of its length by the relationship between moment arms and MT excursions (see Chapter 1, Eq. 1.13) and (2) it removes initial MT length andL T r as a floating parameters. Table 2.2 lists the parameters used. 2.3.3 “Muscle” Dynamics The muscle-like actuators were modelled after the Hill-type muscle model (Eq. 2.7;, Hill, 1953; Brown et al., 1996; Brown and Loeb, 1999; Brown et al., 1999; Song et al., 2008a,b) to include a force-length relationship (f L;i ) and force-velocity relationship (f V;i ) to scale the input (i.e., the “activation”,u i ) as well as a parallel elastic element (F PE;i ) and negligible damping (b m;i ). l m;i = 1 m i f T;i cos( i ) f L;i (l m;i )f V;i l m;i ; _ l m;i u i +F PE 1;i l m;i ; _ l m;i +b m;i _ l m;i f max;i cos 2 ( i ) ! + ( _ l m;i ) 2 tan 2 ( i ) l m;i (2.7) 38 Table 2.2: Musculotendon (MT) parameters used in the tendon tension dynamics (Eq. 2.6d). Max- imum isometric tetanic muscle force (f max;i ), muscle pennation angle ( i ), and optimal tendon length (l T;o;i ) were taken from reported values for the biceps and triceps MTs (combining infor- mation about different heads of the muscle as necessary, Holzbaur et al., 2005). Tendon shape coefficients (c T andk T ) were taken from Brown et al. (1996). Tendon Parameters Parameter “Muscle 1” (Biceps) “Muscle 2” (Triceps) Maximum isometric tetanic muscle force,f max;i (N) 1059.9 2047.1 Muscle pennation angle, i (deg) 0 12 Optimal tendon length,l T;o;i (m) 0.272 0.143 Normalized asymptotic stiffness,c T (unitless) 27.8 27.8 Tendon radius of curvature constant,k T (unitless) 0.0047 0.0047 Note thatm i represents the muscle mass. The equations for the muscle force-length, force-velocity relationships and parallel elastic force (as well as any parameter values) were taken from Song et al. (2008b) and have been reproduced below. Please see Table 2.3, for complete list of parameters used. f L;i (l m;i ) = exp (l m;i =l m;o;i ) 1 ! ; (;;!> 0) (2.8) f V;i l m;i ; _ l m;i = 8 > > > > > > > > < > > > > > > > > : V max _ l m;i l m;o;i V max + c v0 +c v1 l m;i l m;o;i _ l m;i l m;o;i ( _ l m;i 0) b v a v0 +a v1 l m;i l m;o;i +a v2 l m;i l m;o;i 2 _ l m;i l m;o;i b v + _ l m;i l m;o;i ( _ l m;i > 0) (2.9a) (2.9b) F LV;i l m;i ; _ l m;i =f L;i (l m;i )f V;i l m;i ; _ l m;i (2.10) 39 F PE 1;i l m;i ; _ l m;i =c 1 k 1 ln exp l m;i l m;o;i L max CE;i L r1 ! =k 1 + 1 + _ l m;i l m;o;i ! (2.11) Table 2.3: Muscle parameters used by Eq. 2.7. Muscle masses (m i ) and optimal muscle fascicle lengths (l m;o;i ) were taken from reported values for the biceps and triceps glsplMT (combining information about different heads of the muscle as necessary, Holzbaur et al., 2005). Muscle damping (b m;i ) was chosen to be small. Muscle Parameters Parameter “Muscle 1” (Biceps) “Muscle 2” (Triceps) Muscle mass,m i (kg) 0.164 0.325 Optimal muscle fascicle length,l m;o;i (m) 0.116 0.134 Muscle damping,b m;i (kg/s) 0.01 0.01 Table 2.4: Muscle parameters used by Eqs. 2.8–2.11. All coefficients are taken from Song et al. (2008b) from fast twitch muscles, except for L max CE , which was estimated as a floating parameter. Coefficients forEqs. 2.8-2.11 f L Coefficients ! 1.55 0.75 2.12 f V Coefficients (concentric) V max c v0 c v1 -9.15 -5.78 9.18 f V Coefficients (eccentric) a v0 a v1 a v2 b v -1.53 0 0 0.69 F PE 1 Coefficients c 1 k 1 L 1 r L max CE 23.0 0.046 1.17 0.01 1.2 40 2.3.4 Integrator Backstepping Constraint In order to derive the integrator backstepping constraint, we first rewrite these dynamical equa- tions in its state space representation (Eq. 2.13) where~ x = (x 1 ;:::;x 8 ) T is defined by Eq. 2.12. ~ x = ; _ ;f T;1 ;f T;2 ;l m;1 ;l m;2 ; _ l m;1 ; _ l m;2 T (2.12) _ ~ x = 8 > > > > > > > > > > > > > > > > > > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > > > : _ x 1 =x 2 =f 1 (~ x) _ x 2 =c 1 sinx 1 +c 2 r 1 (x 1 )x 3 +c 2 r 2 (x 1 )x 4 =f 2 (~ x) _ x 3 =K T;1 (x 3 ) (v MT;1 (x 1 ;x 2 )c 3 x 7 ) =f 3 (~ x) _ x 4 =K T;2 (x 4 ) (v MT;2 (x 1 ;x 2 )c 4 x 8 ) =f 4 (~ x) _ x 5 =x 7 =f 5 (~ x) _ x 6 =x 8 =f 6 (~ x) _ x 7 =c 5 x 3 c 6 F PE;1 (x 5 ;x 7 )c 7 x 7 + c 8 x 2 7 x 5 c 6 F LV;1 (x 5 ;x 7 )u 1 =f 7 (~ x;~ u) _ x 8 =c 9 x 4 c 10 F PE;2 (x 6 ;x 8 )c 11 x 8 + c 12 x 2 8 x 6 c 10 F LV;2 (x 6 ;x 8 )u 2 =f 8 (~ x;~ u) (2.13a) (2.13b) (2.13c) (2.13d) (2.13e) (2.13f) (2.13g) (2.13h) The constants (c i ’s) were assigned for convenience and are defined in Eqs. 2.14–2.25. c 1 =g=L CM (2.14) c 2 = 1=ML 2 CM (2.15) c 3 = cos( 1 ) (2.16) c 4 = cos( 2 ) (2.17) c 5 = cos( 1 ) m 1 (2.18) 41 c 6 =f max;1 cos 2 ( 1 ) m 1 (2.19) c 7 =f max;1 b m;1 cos 2 ( 1 ) m 1 l m;o;1 (2.20) c 8 = tan 2 ( 1 ) (2.21) c 9 = cos( 2 ) m 2 (2.22) c 10 =f max;2 cos 2 ( 2 ) m 2 (2.23) c 11 =f max;2 b m;2 cos 2 ( 2 ) m 2 l m;o;2 (2.24) c 12 = tan 2 ( 2 ) (2.25) Consider the output of the system to be the joint angle (x 1 , Eq. 2.26) which will be made to follow some reference trajectory ( r ). Then the deviation of the joint angle from the desired reference trajectory can be defined by Eq. 2.27. y =x 1 (2.26) z 1 = r x 1 (2.27) _ z 1 = _ r _ x 1 = _ r x 2 (2.28a) (2.28b) Next, we propose the Lyapunov function candidate: V 1 = 1 2 z 2 1 (2.29) 42 such that, _ V 1 =z 1 _ z 1 =z 1 ( _ r x 2 ) (2.30) To address the stability of this difference variable, we must show that _ V 1 is negative definite ( _ V 1 < 0 for allz 1 6= 0 and _ V 1 (0) = 0) so that for allz 1 6= 0 the difference variablez 1 will be approaching zero for all time. This can be done by considering the statex 2 to be a “virtual input” to the upstream statex 1 . But because we do not have direct control over whatx 2 will do, we define the value that we would like it to approach ( 1 , Eq. 2.31). By choosingk 1 to be greater than 0, _ V 1 is negative definite inz 1 . x 2 ! 1 _ r +k 1 z 1 ; k 1 > 0 (2.31) Similar to what was done whenx 1 was tracking r , we define a second difference variable (z 2 , Eq. 2.32), suggest a Lyapunov function (Eq. 2.34), and find what can make its time derivative (Eq. 2.35) negative definite to ensure the difference always approaches zero. z 2 =x 2 1 (2.32) _ z 2 = _ x 2 _ 1 =f 2 (~ x) _ 1 =c 1 sinx 1 +c 2 r 1 (x 1 )x 3 +c 2 r 2 (x 1 )x 4 _ 1 (2.33a) (2.33b) (2.33c) V 2 =V 1 + 1 2 z 2 2 (2.34) 43 _ V 2 = _ V 1 +z 2 _ z 2 =z 1 ( _ r 1 z 2 ) +z 2 (c 1 sinx 1 +c 2 r 1 (x 1 )x 3 +c 2 r 2 (x 1 )x 4 _ 1 ) =z 1 ( _ r 1 ) +z 2 (c 1 sinx 1 +c 2 r 1 (x 1 )x 3 +c 2 r 2 (x 1 )x 4 _ 1 z 1 ) (2.35a) (2.35b) (2.35c) By definition of 1 (Eq. 2.31), the first term in Eq. 2.35c is negative definite. Therefore _ V 2 is negative definite (and the difference valuesz 1 andz 2 are guaranteed to decay to zero) if downstream statesx 3 andx 4 are made to follow a new reference value ( 2 , Eq. 2.36). c 2 r 1 (x 1 )x 3 +c 2 r 2 (x 1 )x 4 ! 2 _ 1 +z 1 c 1 sinx 1 k 2 z 2 (2.36) Moving down the chain of states, this process is repeated again another difference variable (z 3 , Eq. 2.37). z 3 =c 2 r 1 (x 1 )x 3 +c 2 r 2 (x 1 )x 4 2 (2.37) _ z 3 =c 2 r 0 1 (x 1 ) _ x 1 x 3 +c 2 r 1 (x 1 ) _ x 3 +c 2 r 0 2 (x 1 ) _ x 1 x 4 +c 2 r 2 (x 1 ) _ x 4 _ 2 (2.38) =c 2 r 0 1 (x 1 )f 1 (~ x)x 3 +c 2 r 1 (x 1 )f 3 (~ x) +c 2 r 0 2 (x 1 )f 1 (~ x)x 4 +c 2 r 2 (x 1 )f 4 (~ x) _ 2 (2.39) =c 2 r 0 1 (x 1 )f 1 (~ x)x 3 +c 2 r 0 2 (x 1 )f 1 (~ x)x 4 _ 2 +c 2 r 1 (x 1 )[K T;1 (x 3 )v MT;1 (x 1 ;x 2 )c 3 K T;1 (x 3 )x 7 ] (2.40) +c 2 r 2 (x 1 )[K T;2 (x 4 )v MT;2 (x 1 ;x 2 )c 4 K T;2 (x 4 )x 8 ] _ z 3 =c 2 r 0 1 (x 1 )f 1 (~ x)x 3 +c 2 r 0 2 (x 1 )f 1 (~ x)x 4 _ 2 +c 2 r 1 (x 1 )K T;1 (x 3 )v MT;1 (x 1 ;x 2 ) +c 2 r 2 (x 1 )K T;2 (x 4 )v MT;2 (x 1 ;x 2 ) (2.41) c 2 c 3 r 1 (x 1 )K T;1 (x 3 )x 7 c 2 c 4 r 2 (x 1 )K T;2 (x 4 )x 8 44 V 3 =V 2 + 1 2 z 2 3 (2.42) _ V 3 = _ V 2 +z 3 _ z 3 =z 1 ( _ r 1 ) +z 2 (c 1 sinx 1 _ 1 z 1 + 2 +z 3 ) +z 3 _ z 3 =z 1 ( _ r 1 ) +z 2 (c 1 sinx 1 _ 1 z 1 + 2 ) +z 3 (z 2 + _ z 3 ) (2.43a) (2.43b) (2.43c) By the definitions of 1 and 2 , the first two terms of Eq. 2.43c are negative definite. Therefore, the difference variables will converge to zero if, z 2 + _ z 3 =k 3 z 3 ;k 3 > 0 (2.44) This can be accomplished ifx 7 andx 8 are made to follow a new reference value ( 3 , Eqs. 2.45- 2.46). c 2 c 3 r 1 (x 1 )K T;1 (x 3 )x 7 +c 2 c 4 r 2 (x 1 )K T;2 (x 4 )x 8 ! 3 (2.45) 3 k 3 z 3 +z 2 _ 2 +c 2 r 0 1 (x 1 )f 1 (~ x)x 3 +c 2 r 0 2 (x 1 )f 1 (~ x)x 4 +c 2 r 1 (x 1 )K T;1 (x 3 )v MT;1 (x 1 ;x 2 ) +c 2 r 2 (x 1 )K T;2 (x 4 )v MT;2 (x 1 ;x 2 ) (2.46) This process is completed one more time to reach the input level of the state space. We in- troduce the last difference variable (z 4 ), Lyapunov function candidate, and the constraint on the inputs, ~ u that will guarantee all difference variables (z 1 ;:::z 4 ) asymptotically converge to zero 45 producing stable reference tracking of the original trajectory, r . z 4 =c 2 c 3 r 1 (x 1 )K T;1 (x 3 )x 7 +c 2 c 4 r 2 (x 1 )K T;2 (x 4 )x 8 3 (2.47) _ z 4 =c 2 c 3 r 0 1 (x 1 ) _ x 1 K T;1 (x 3 )x 7 +c 2 c 3 r 1 (x 1 )K 0 T;1 (x 3 ) _ x 3 x 7 +c 2 c 3 r 1 (x 1 )K T;1 (x 3 ) _ x 7 +c 2 c 4 r 0 2 (x 1 ) _ x 1 K T;2 (x 4 )x 8 (2.48) +c 2 c 4 r 2 (x 1 )K 0 T;2 (x 4 ) _ x 4 x 8 +c 2 c 4 r 2 (x 1 )K T;2 (x 4 ) _ x 8 _ 3 =c 2 c 3 r 0 1 (x 1 )f 1 (~ x)K T;1 (x 3 )x 7 +c 2 c 3 r 1 (x 1 )K 0 T;1 (x 3 )f 3 (~ x)x 7 +c 2 c 3 r 1 (x 1 )K T;1 (x 3 )f 7 (~ x) +c 2 c 4 r 0 2 (x 1 )f 1 (~ x)K T;2 (x 4 )x 8 (2.49) +c 2 c 4 r 2 (x 1 )K 0 T;2 (x 4 )f 4 (~ x)x 8 +c 2 c 4 r 2 (x 1 )K T;2 (x 4 )f 8 (~ x) _ 3 =c 2 c 3 r 0 1 (x 1 )f 1 (~ x)K T;1 (x 3 )x 7 +c 2 c 3 r 1 (x 1 )K 0 T;1 (x 3 )f 3 (~ x)x 7 +c 2 c 4 r 0 2 (x 1 )f 1 (~ x)K T;2 (x 4 )x 8 +c 2 c 4 r 2 (x 1 )K 0 T;2 (x 4 )f 4 (~ x)x 8 _ 3 +c 2 c 3 r 1 (x 1 )K T;1 (x 3 ) h c 5 x 3 c 6 F PE;1 (x 5 ;x 7 )c 7 x 7 (2.50) + c 8 x 2 7 x 5 c 6 F LV;1 (x 5 ;x 7 )u 1 i +c 2 c 4 r 2 (x 1 )K T;2 (x 4 ) h c 9 x 4 c 10 F PE;2 (x 6 ;x 8 )c 11 x 8 + c 12 x 2 8 x 6 c 10 F LV;2 (x 6 ;x 8 )u 2 i =c 2 c 3 r 0 1 (x 1 )f 1 (~ x)K T;1 (x 3 )x 7 +c 2 c 3 r 1 (x 1 )K 0 T;1 (x 3 )f 3 (~ x)x 7 +c 2 c 4 r 0 2 (x 1 )f 1 (~ x)K T;2 (x 4 )x 8 +c 2 c 4 r 2 (x 1 )K 0 T;2 (x 4 )f 4 (~ x)x 8 _ 3 +c 2 c 3 r 1 (x 1 )K T;1 (x 3 ) c 5 x 3 c 6 F PE;1 (x 5 ;x 7 )c 7 x 7 + c 8 x 2 7 x 5 +c 2 c 4 r 2 (x 1 )K T;2 (x 4 ) c 9 x 4 c 10 F PE;2 (x 6 ;x 8 )c 11 x 8 + c 12 x 2 8 x 6 (2.51) c 2 c 3 c 6 r 1 (x 1 )K T;1 (x 3 )F LV;1 (x 5 ;x 7 )u 1 c 2 c 4 c 10 r 2 (x 1 )K T;2 (x 4 )F LV;2 (x 6 ;x 8 )u 2 46 V 4 =V 3 + 1 2 z 2 4 (2.52) Such that, _ V 4 = _ V 3 +z 4 _ z 4 =z 1 ( _ r 1 ) +z 2 (c 1 sinx 1 _ 1 z 1 + 2 ) +z 3 z 2 _ 2 +c 2 r 0 1 (x 1 )f 1 (~ x)x 3 +c 2 r 0 2 (x 1 )f 1 (~ x)x 4 +c 2 r 1 (x 1 )K T;1 (x 3 )v MT;1 (x 1 ;x 2 )z 4 +c 2 r 2 (x 1 )K T;2 (x 4 )v MT;2 (x 1 ;x 2 ) 3 +z 4 _ z 4 =z 1 ( _ r 1 ) +z 2 (c 1 sinx 1 _ 1 z 1 + 2 ) +z 3 z 2 _ 2 +c 2 r 0 1 (x 1 )f 1 (~ x)x 3 +c 2 r 0 2 (x 1 )f 1 (~ x)x 4 +c 2 r 1 (x 1 )K T;1 (x 3 )v MT;1 (x 1 ;x 2 ) +c 2 r 2 (x 1 )K T;2 (x 4 )v MT;2 (x 1 ;x 2 ) 3 +z 4 ( _ z 4 z 3 ) (2.53a) (2.53b) (2.53c) By the definitions of 1 , 2 , and 3 , the first three terms of Eq. 2.53c are negative definite. Therefore, stability of the entire system (from the asymptotic stability for allz i ) can be obtained if, _ z 4 z 3 =k 4 z 4 ;k 4 > 0 (2.54) which creates the constraint on the input given by Eq. 2.55. 47 c 2 c 3 c 6 r 1 (x 1 )K T;1 (x 3 )F LV;1 (x 5 ;x 7 )u 1 + c 2 c 4 c 10 r 2 (x 1 )K T;2 (x 4 )F LV;2 (x 6 ;x 8 )u 2 =c 2 c 3 r 0 1 (x 1 )f 1 (~ x)K T;1 (x 3 )x 7 (2.55) +c 2 c 3 r 1 (x 1 )K 0 T;1 (x 3 )f 3 (~ x)x 7 +c 2 c 4 r 0 2 (x 1 )f 1 (~ x)K T;2 (x 4 )x 8 +c 2 c 4 r 2 (x 1 )K 0 T;2 (x 4 )f 4 (~ x)x 8 +c 2 c 3 c 5 r 1 (x 1 )K T;1 (x 3 )x 3 c 2 c 3 c 6 r 1 (x 1 )K T;1 (x 3 )F PE;1 (x 5 ;x 7 ) c 2 c 3 c 7 r 1 (x 1 )K T;1 (x 3 )x 7 +c 2 c 3 c 8 r 1 (x 1 )K T;1 (x 3 ) x 2 7 x 5 +c 2 c 4 c 9 r 2 (x 1 )K T;2 (x 4 )x 4 c 2 c 4 c 10 r 2 (x 1 )K T;2 (x 4 )F PE;2 (x 6 ;x 8 ) c 2 c 4 c 11 r 2 (x 1 )K T;2 (x 4 )x 8 +c 2 c 4 c 12 r 2 (x 1 )K T;2 (x 4 ) x 2 8 x 6 _ 3 z 3 +k 4 z 4 This equation produces one equation to constrain two inputs. In order to resolve the redundancy in this system, another constraint must be provided. By forcing the pendulum to follow a simple 1 Hz sinusoidal reference trajectory ( r (t) = 7:5 cos(2t) + 90 in degrees) with this integrator backstepping constraint and by choosing one of the input signals (u 1 ) to be a 1 Hz signal with variable amplitude, offset, and/or phase shift, it is possible to systematically sample the nullspace of this redundant system (Figure 2.4). This choice of “muscle activations” may not be exactly what 48 Nature would choose and is generally sub-optimal by definition for most cost functions, but this control example resolves redundancy in a systematic way to explore the effect of “sub-optimal” movement commands like these have on how the resulting muscle mechanics relate to the MT behavior with respect to the induced tendon tensions. Lastly, it is important to discuss how these simulations are initialized. As with any forward in- tegration problem, the initial value is critical for the behavior of the system. This is especially im- portant for systems with muscle-like actuators because the choice of initial muscle fascicle length and initial muscle activation will be influenced by the level of pretensioning in the system which is, by definition, redundant. That is to say when the pendulum begins from rest (i.e., = _ = 0), there are an infinite number of tendon forces that produce the net joint torque needed to negate the gravitational torque (Eq. 2.56), implying that there are an infinite number of valid initial muscle fascicle lengths and activations. If we assume that the muscle states are also in equilibrium (i.e., l m;i = _ l m;i = 0), there (intuitively) exists an explicit relationship between the initial activation, the amount of pretensioning on the tendon, and the initial muscle fascicle length (Eqs. 2.57–2.58). LCM M θ Desired θ (Deg) 4s f T,1 f T,2 r 2 r 1 0 2 4 Time (s) 0.2 0.1 Muscle Activation u i (A) (B) (C) (D) 90 97.5 82.5 Angle θ (Deg) 40 0 -40 Angular Velocity θ (Deg/s) . Tendon Tension f T,i (N) 300 200 100 120 100 80 Muscle Length lm,i (mm) 40 0 -40 Muscle Velocity vm,i (mm/s) (E) Figure 2.4: Example of the integrator backstepping approach to sampling the nullspace of a com- pliant tendon-driven system (A,B), made to follow a reference trajectory (C), by prescribing the activation of one actuator (D). The resulting states are shown in the bottom panel (E). 49 For initial muscle fascicle lengths near the optimal length, the range of initial activations that produces the initial equilibrium tendon tension is quite small (near zero slope in Figure 2.5 when l m;i l m;o;i ). However, when the initial muscle fascicle lengths becomes smaller (l m;i <l m;o;i ), the muscle moves down the ascending curve of its force-length relationship (F L ) which dominates the denominator thereby requiring larger initial muscle activations to produce the same initial tendon tension. Alternatively, when the muscle becomes increasingly stretched (l m;i >l m;o;i ), the passive elasticity of the muscle dominates (F PE 1 ), requiring less initial muscle activation to produce the same initial tendon tension. g L CM sin ((t o )) =r 1 ((t o ))f T;1 (t o )r 2 ((t o ))f T;2 (t o ) (2.56) f T;i (t o ) =f max;i cos( i ) f L;i l m;i (t o ) u i (t o ) +F PE 1 l m;i (t o ); 0 (2.57) u(t o ) = f T;i (t o ) f max;i cos( i ) F PE 1 l m;i (t o ); 0 f L;i l m;i (t o ) (2.58) The previous chapter compared similar movements to understand the effect that kinematic redundancy has on MT behavior. By introducing the dynamics of tendon and a controller into the problem, we can now compare identical movements to see how changes in pretensioning or activation levels alters the relationship between MT and muscle fascicles. To this end, we will perform tasks where either the initial tensions or the initial muscle fascicle lengths are fixed while systematically changing the prescribed input signal (u 1 ). 50 f T,i (t o )= 20% f max,i f T,i (t o )= 10% f max,i 0.3 0.5 0.1 0.7 0.5 0.75 1.0 1.25 F PE,1 (l m,i (t o ),0) Dominates F L (l m,i (t o )) Dominates Initial Normalized Muscle Fascicle Length Initial Muscle Activation Figure 2.5: Relationship between initial muscle fascicle lengths and initial muscle activations for a given initial equilibrium tendon tension. Note that for initial muscle fascicle lengths near the optimal muscle fascicle length the range of compatible initial activations is quite small. However, when the muscle fascicle length becomes smaller (l m;i < l m;o;i ), the muscle is on the descend- ing curve of its force-length relationship (F L ) which dominates the denominator causing larger required activations to produce the same tendon tension. Alternatively, when the muscle becomes increasingly stretched (l m;i >l m;o;i ), the passive elasticity of the muscle dominates (F PE 1 ), requir- ing less muscle activation to produce the same initial tendon tension. 2.4 Results 2.4.1 Fixed Initial Muscle Fascicle Lengths To determine the effect, if any, the level of pretensioning has on the relationship between MT and muscle behavior, a 15 sinusoidal joint movement simulation was performed where the initial muscle fascicle lengths were held constant while the initial tensions were varied. To minimize integration errors associated with transitioning from (near) rest conditions to a continuously differ- entiable sinusoidal reference trajectory, the movement was chosen to begin in the flexed position (97.5 ) and follow a cosine trajectory (extending down to 82.5 before returning to the flexed po- sition). As such, it was reasonably determined from the size of their respective moment arms and the amplitude of the movement that the initial muscle fascicle lengths should be slightly short- 51 ened for muscle 1 (the flexor, 0.95 l m;o;1 ) and lengthened muscle 2 (the extensor, 1.025 l m;o;2 ). Initial tendon tensions were chosen by selecting 10 equidistant points along the initial equilibrium constraint (Eq. 2.56) subject to 0:15f max;i f T;i 0:5f max;i for i2f1; 2g. From Figure 2.5, we expect that for any fixed initial muscle fascicle length, increasing initial tendon tension will increase the resulting initial muscle activation. Figure 2.6 shows the resulting states for all 10 82.5 90.0 97.5 θ (deg) 0 1 2 3 4 Time (s) −40 −20 0 20 40 θ (deg/s) . 100 200 300 400 500 f T,1 (N) 1 0.95 1.05 −0.25 0 0.25 0 200 400 600 800 1000 f T,2 (N) 0 1 2 3 4 Time (s) 0.975 1 1.025 0 0.25 v m,1 (l m,o,1 /s) ^ l m,1 (l m,o,1 ) ^ v m,2 (l m,o,2 /s) ^ l m,2 (l m,o,2 ) ^ Figure 2.6: Sweeping tendon pretensioning levels. For 10 different baseline tensions an integrator backstepping controller was used to force a pendulum controlled by two muscle-like actuators that pull on compliant tendons to follow a sinusoidal trajectory. For this simulation, the initial muscle fascicle lengths were fixed for each trial while the initial tendon tensions were selected such that they satisfy Eq. 2.56 and 0:15f max;i f T;i 0:5f max;i for i 2 f1; 2g. Muscle lengths and velocities have been normalized by their respective optimal muscle fascicle lengths (l m;o;i ). 52 0 1 2 3 4 Time (s) 0 0.10 0.20 0.30 Error (Deg) Figure 2.7: Error between reference trajectory ( r (t)) and the actual joint angle () (controlled via an integrator backstepping approach) when pretensioning is systematically varied. All 10 trials are superimposed on top of one another, so it is clear that the performance was not affected by the choice of initial tendon tension. For this simulation, initial muscle fascicle lengths were fixed while the initial tendon tensions were selected such that they satisfy Eq. 2.56 and 0:15f max;i f T;i 0:5f max;i fori2f1; 2g. initial tendon tension settings while Figure 2.7 shows the performances of the controller of all 10 trials (i.e., r (t)). The control errors for these 10 trials are (i) very small ( 0:33 ) and (ii) consistent across different choices of initial tendon tension (i.e., pretensioning does not affect the performance of the controller). Figure 2.8 compares the MT velocities to muscle fascicle velocities (all normalized by the optimal muscle fascicle length). Previous work equated these values but ignored the effect of com- pliant tendons and the pennation angle of the muscle. The error between these two value (Figure 2.8 right) shows that these values not only differ but also that the pretensioning (as well as the subsequent time histories of tendon tensions) affect this difference. Interestingly, approximating muscle fascicle velocity as MT velocity incurs larger errors for higher pretensioning for muscle 1, while the opposite is seen in muscle 2. This may be due to the effects of gravity as muscle 1 must compete with the additional gravitational torques. We extend this analysis to the assumption that MT excursions (i.e., change in MT length) can approximate muscle fascicle length changes (Figure 2.9). Consistent with the data shown in Figure 2.8, we see an overestimate of the magnitude of muscle fascicle length change when there is high 53 pretensioning in muscle 1, but also when there is low pretensioning in muscle 2. If we plot the mean absolute error of the comparison between MT excursion and muscle fascicle length change (normalized by optimal muscle fascicle length), we see these trends more clearly (Figure 2.10). This relationship does not appear to be linear, but does appear to be well defined, implying that there may exist an explicit form for the difference between MT and muscle behavior. −0.25 0 0.25 −0.05 0 0.05 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Time (s) 0.0 0.25 0 0.10 0.20 0.30 Normalized Muscle Velocity Normalized Musculotendon Velocity Low High f T,1 (t o ) Low High f T,2 (t o ) v m,1 (l m,o,1 /s) ^ v m,2 (l m,o,2 /s) ^ Normalized Error v m,i - v MT,i /l m,o,i (in l m,o,i /s) ^ Figure 2.8: Comparing normalized muscle fascicle velocities to the MT velocities associated with the movement (normalized by optimal muscle fascicle lengths) for muscles 1 (blue) and 2 (red) for 10 trials with different levels of pretensioning. The error between these two measurements is explicitly plotted on the right. Clearly, the assumption that MT velocity can be used to approximate muscle fascicle velocity will incur tension and muscle specific errors. 54 0.05 0 0.1 −0.005 0 0.005 0.01 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Time (s) -0.05 -0.025 0 0 0.005 0.01 0.015 Normalized Muscle Length Change Normalized Musculotendon Excursion Low High f T,1 (t o ) Low High f T,2 (t o ) ∆l m,1 (l m,o,1 ) ^ Normalized Error ∆l m,i - ∆l MT,i /l m,o,i (in l m,o,i ) ^ ∆l m,2 (l m,o,2 ) ^ Figure 2.9: Comparing normalized muscle fascicle length changes (with respect to the initial mus- cle fascicle length) to the MT excursions (i.e., length changes) associated with the movement (normalized by optimal muscle fascicle lengths) for muscles 1 (blue) and 2 (red) for 10 trials with different levels of pretensioning. The error between these two measurements is explicitly plotted on the right. Consistent with the data seen in Figure 2.8, increasing the level of pretensioning (and as a result, the average amount of tension on each tendon) appears to increase the magnitude of the error for muscle 1, but decreases the the magnitude of the error for muscle 2. Clearly, the assumption that MT excursions can be used to approximate muscle fascicle length changes (and by extension, muscle fascicle lengths themselves) may incur tension and muscle specific errors. 55 15 25 35 45 Initial Normalized Tendon Force (f T,1 (t o ) as %f max,i ) 0.4 0.5 0.6 0.7 Mean Absolute Error (%l m,o,i ) Muscle 1 20 30 40 50 Muscle 2 Figure 2.10: Plotting the mean absolute error between the (ground truth) muscle fascicle length changes and the MT excursions (expressed as a percentage of the optimal muscle fascicle length) as a function of the level of pretensioning on the tendons. These results are consistent with the trends seen in Figures 2.8 & 2.9 where we see a positive relationship between the error and the pretensioning for muscle 1, but a negative relationship for muscle 2. This relationship does not appear to be linear, but does appear to be well defined, implying that an explicit form of the error may exist. 2.4.2 Fixed Initial Tendon Tensions The force of mammalian skeletal muscle depends on its current length and velocity (Blix, 1894; Stevens and Snodgrass, 1932; Joyce et al., 1969; Rack and Westbury, 1969; Hatcher and Luff, 1986; Gareis et al., 1992; Herzog et al., 1992). Therefore, producing the same movement when starting from different initial muscle fascicle lengths will require different muscle mechanics and activations that may affect the relationship between MT and muscle behavior. To that end, the same sinusoidal joint movement from Section 2.4.1 (15 sinusoidal) was simulated with uniformly sampled initial muscle fascicle lengths (assuming constant pretensioning value). Figure 2.11 shows the resulting states for 25 different trials where initial muscle fascicle lengths were varied while Figure 2.12 shows the performances of the controller for all 25 trials (i.e., r (t)). Similar to the previous simulation, the control errors for these 25 trials are (i) very small ( 0:33 ) and (ii) 56 consistent across different choices of initial muscle fascicle lengths. Figure 2.13 compares the MT velocities to muscle fascicle velocities normalized to the optimal muscle fascicle length. While it is clear that an error between these two value exists (Figure 2.13 right), it does not appear to be affected by the initialization of the muscle fascicle lengths. 82.5 90.0 97.5 θ (deg) 0 1 2 3 4 Time (s) −40 −20 0 20 40 θ (deg/s) . f T,1 (N) 200 300 400 −0.25 0 0.25 v m,1 (l m,o,1 /s) ^ 0.9 0.8 1 1.1 l m,1 (l m,o,1 ) ^ 200 400 600 800 f T,2 (N) 0 1 2 3 4 Time (s) 0 0.25 v m,2 (l m,o,2 /s) ^ 1 1.2 0.8 0.6 l m,2 (l m,o,2 ) ^ Figure 2.11: Sampling initial muscle fascicle lengths. For 25 different initial muscle fascicle lengths an integrator backstepping controller was used to force a pendulum controlled by two muscle-like actuators that pull on compliant tendons to follow a sinusoidal trajectory. For this simulation, the initial tendon tensions were fixed for each trial while the initial muscle fascicle lengths were uniformly selected such that they satisfy Eq. 2.56. Muscle lengths and velocities have been normalized by their respective optimal muscle fascicle lengths (l m;o;i ). 57 0 1 2 3 4 Time (s) 0 0.10 0.20 0.30 Error (Deg) Figure 2.12: Error between reference trajectory ( r (t)) and the actual joint angle () (controlled via an integrator backstepping approach) when initial muscle fascicle lengths are uniformly sampled. All 25 trials are superimposed on top of one another, so it is clear that the performance was not affected by the choice of initial muscle fascicle lengths. For this simulation, initial tendon tensions were fixed while the initial muscle fascicle lengths were uniformly sampled subject to Eqs. 2.57- 2.58. −0.25 0 0.25 −0.10 0 0.10 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Time (s) 0.0 0.25 0.00 0.10 0.20 v m,1 (l m,o,1 /s) ^ v m,2 (l m,o,2 /s) ^ Low High l m,1 (t o ) Low High l m,2 (t o ) Normalized Error v m,i - v MT,i /l m,o,i (in l m,o,i /s) ^ Normalized Muscle Velocity Normalized Musculotendon Velocity Figure 2.13: Comparing normalized muscle fascicle velocities to the MT velocities associated with the movement (normalized by optimal muscle fascicle lengths) for muscles 1 (blue) and 2 (red). The error between these two measurements is explicitly plotted on the right. While there clearly exists a difference between MT and muscle fascicle velocities, it does not appear that any obvious relationship exists between the initial muscle fascicle length and this difference. 58 0.05 0 0.1 −0.01 0 0.01 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Time (s) -0.05 -0.025 0 0 0.005 0.01 0.015 ∆l m,1 (l m,o,1 ) ^ Normalized Error ∆l m,i - ∆l MT,i /l m,o,i (in l m,o,i ) ^ Normalized Muscle Length Change Normalized Musculotendon Excursion ∆l m,2 (l m,o,2 ) ^ Low High l m,1 (t o ) Low High l m,2 (t o ) Figure 2.14: Comparing normalized muscle fascicle length changes (with respect to the initial tendon tensions) to the MT excursions (i.e., length changes) associated with the movement (nor- malized by optimal muscle fascicle lengths) for muscles 1 (blue) and 2 (red). The error between these two measurements is explicitly plotted on the right. Consistent with the data seen in Fig- ure 2.8, there are differences between MT excursions and muscle fascicle length changes, but the differences do not appear to depend on the choice of initial muscle fascicle length. This trend is also observed when MT excursions are compared to muscle fascicle length changes (Figure 2.14). However, it should be noted that a slight trend may exist between the error in mus- cle 1 and the choice of its initial muscle fascicle length that is not present in muscle 2. This can be seen more clearly when comparing the mean absolute error of the comparison between MT excursion and muscle fascicle length changes as a function of the initial muscle fascicle lengths (Figure 2.15). This is an artifact of the way in which the activation for muscle 1 was prescribed. By uniformly sampling initial muscle fascicle lengths, initial activations are found by solving Eq. 2.58. Sampling from a reasonable range of initial muscle fascicle lengths results in a small range of initial muscle activations sampled from the flat region of Figure 2.5 (i.e., l m;1 l m;o;1 ). The activation signal for muscle 1 is then prescribed to be a slightly phase-delayed cosine wave with 59 constant amplitude. Therefore, there is not a considerable difference between the activation sig- nals for muscle 1 across trials, even though initial muscle fascicle lengths vary between 0.8 and 1.0l m;o;1 . This means that muscle 1 will produce slightly less tension (because of the ascending curve of the force-length relationship, Eq. 2.8), when initialized at a shorter initial muscle fascicle length forcing muscle 2 to compensate through the integrator backstepping constraint (Eq. 2.55) to produce the desired movement. And as discussed in Section 2.4.1, the difference between MT and muscle fascicle behavior appears to depend on tendon tension. Muscle 2 Mean Absolute Error (%l m,o,i ) Initial Normalized Muscle Length (l m,i (t o ) as %l m,o,i ) Muscle 1 75 80 85 90 95 100 0.5 0.6 0.7 0.8 0.9 1.0 60 80 100 120 Figure 2.15: Plotting the mean absolute error between the (ground truth) muscle fascicle length changes and the MT excursions (expressed as a percentage of the optimal muscle fascicle length) as a function of initial muscle fascicle lengths. These results are consistent with the trends seen in Figures 2.8 & 2.9 for muscle 2, but we do see a slight positive trend for muscle 1. 2.5 Discussion The interaction between muscle and tendon required to produce smooth movements is anything but simple. An important consequence of muscle redundancy is that the same movement can be accomplished with an infinite combination of tendon tensions, so long as they produce the same net joint torques. But this redundancy at the tendon level translates to redundancy at the muscle level, where it can be equivalently stated that for the same movement there are an infinite number of 60 choices for muscle activation that exist that each produce their own unique muscle fascicle lengths and velocities so long as they stretch the tendon enough to produce the required tension. Therefore, it is impossible to infer muscle (or tendon) behavior based solely on joint kinematics because choices from the nullspace of equivalent tendon tensions require different muscle mechanics. And while it is useful to approximate muscle fascicle lengths and velocities from kinematically-derived MT behaviors, the degree to which this approximation deviates from the true underlying muscle behavior needs to be explored. In this chapter we begin to elucidate the role that compliant tendon plays in the relationship between muscle mechanics and MT behavior (and by extension, joint kinematics). By explor- ing the tendon tension nullspace that exists for this simple redundant tendon-driven system, it was demonstrated that muscle fascicle lengths and velocities deviate from MT behaviors. Perhaps more importantly, it was determined that the differences between MT and muscle behaviors appear to be (i) tendon tension specific and (ii) nonlinear. This is a direct consequence of nonlinear tendon elasticity and the nullspace of the task—i.e., the same movement can require completely different tendon tensions which must be produced by muscles that change their lengths appropriately to pull on compliant tendons. While the difference between MT and muscle behaviors was not signifi- cantly large for such a simple movement task that required low tendon forces (and small tendon force derivatives), it will be important to quantify the relationship between muscle fascicle lengths and joint kinematics as changes in movement dynamics or tendon elasticity may further decouple muscle and MT. 61 Chapter 3 Accurate musculotendon modeling requires tendon tension... some- times 3.1 Abstract Accurate predictions of tendon forces must consider musculotendon (MT) mechanics, and specifically muscle fascicle lengths and velocities. This is predicted explicitly by simulating the dynamical equations of the musculoskeletal system, or approximated from measured limb kinemat- ics. The latter is complicated by the fact that tendon lengths and pennation angles vary with both limb kinematics and tendon tension. Therefore, approximating muscle mechanics from joint kine- matics does not capture the true behavior of a tendon-driven system, or the relationship between kinematic states and actuator states. We now quantify the error in kinematically-approximated muscle fascicle lengths as a general function of muscle geometry and tendon tension. This equa- tion enables researchers to objectively evaluate the significance of this error in muscle fascicle lengths—which we find can be as high as 80% of the optimal muscle fascicle length—with respect to the scientific or clinical question being asked. Although this equation provides a de- tailed functional relationship between muscle fascicle length and tendon tension, the parameters used to characterize MT architecture are subject-, muscle-, and potentially rate-specific. There- fore, uncertainty in these parameters limits the accuracy of any generic musculoskeletal model that hopes to explain subject-specific phenomena. Alternatively, these computational hurdles have profound implications to proprioception. In particular, they argue that muscle spindles cannot, on their own, reliably reflect joint kinematics. This points to the necessity of knowing tendon tensions in addition to muscle fascicle lengths to accurately estimate joint angles, which strongly suggests that Golgi tendon organs compliment muscle spindles to provide more accurate estimates of body 62 l m ρ l m cos( ρ) l MT = l T + l m cos(ρ) l T,1 l T,2 l m (t o ) l m (t) ρ(t o ) ρ(t) B A Figure 3.1: Approximation of musculotendon (MT) geometry as a flattened parallel bundle of pennated muscle fascicles in series with tendon (A) such that the change in length or excursion (l MT , B) is defined as the sum of tendon length change (combining the tendons of origin and insertion, or l T = l T;1 + l T;2 ) and change in the portion of muscle fascicle length projected onto the line of action of the MT (l m (t) cos((t))l m (t o ) cos((t o )), Gans and Bock, 1965; Gans, 1982; Zajac, 1989). configurations to enable effective control of movement. 3.2 Introduction Musculotendon (MT) complexes, as the name suggests, are composed of both muscle fascicles and tendons (Zajac, 1989). These three-dimensional (often overlapping) bundles of muscle fas- cicles bulge, twist, and change lengths during contractions, but are conceptualized for simplicity as a flattened parallel bundle of muscle fascicles in series with an elastic element. This parallelo- gram simplification states that all muscle fascicles act in parallel but askew from the line of action by some pennation angle, . This compartmentalizes the contributions of muscle fascicle length (l m ) and tendon length (l T ) to MT behavior (Figure 3.1, Gans and Bock, 1965; Gans, 1982; Zajac, 1989) 5 . Thus, MT length (l MT ) is defined as the sum of tendon length (combining the tendons of origin and insertion, or l T = l T;1 +l T;2 ) and muscle fascicle length (l m ) projected onto the line-of-action of the MT (Figure 3.1A, Eq. 3.1, Zajac, 1989). l MT (t) =l T (t) +l m (t) cos((t)) (3.1) 5 See Appendix B for implications of ignoring muscle “bulging” in this simplified parallelogram model. 63 The excursion of the MT (l MT ) at any point in time is given by the difference between MT lengths at timest andt o (Eq. 3.2), such thatl m at timet is given by Eq. 3.3. l MT (t) = l T (t) +l m (t) cos((t))l m (t o ) cos((t o )) (3.2) l m (t) = 1 cos((t)) l MT (t) l T (t) + cos((t o )) cos((t)) l m (t o ) (3.3) In practice, MT excursion can be approximated from the measured kinematics of a movement by tracing the tendon routing from its origin to insertion across postures (Grieve, 1978), or by calculating the integral of the moment arm functions of all joints crossed (Valero-Cuevas, 2016; Kurse et al., 2012; An et al., 1983). However, values for l m , l T , and are difficult to measure in vivo for a given subject, so databases of average values from imaging or cadaver studies are often used as approximations (Rankin and Neptune, 2012; Menegaldo et al., 2004; Grieve, 1978; Arampatzis et al., 2006; Muraoka et al., 2005; Roy and Edgerton, 1992; Lichtwark and Wilson, 2008; Magnusson et al., 2001; Ward et al., 2009; Amis et al., 1979; An et al., 1981; Brand et al., 1981; Edgerton et al., 1990; Friederich and Brand, 1990; Huijing, 1985; Jacobson et al., 1992; Lieber and Brown, 1992; Scott et al., 1993; Wickiewicz et al., 1983; Yamaguchi et al., 1990). This makes kinematics-based approximations for muscle fascicle lengths the most practical and most commonly used. These approximations further rely on the fundamental assumptions that tendons are inextensible (i.e., l T (t) 0) or that is either constant ((t) c ) or negligible ((t) 0). When only assuming inextensible tendons (IT ), Eq. 3.3 simplifies to: ~ l IT m (t) l MT (t) cos((t)) + cos((t o )) cos((t)) l m (t o ) (3.4) Alternatively, in the case where tendon stretch is included but is assumed constant (CP ), Eq. 3.3 64 simplifies to: ~ l CP m (t) 1 cos( c ) l MT (t) l T (t) +l m (t o ) (3.5) More often, however, both assumptions are made and the kinematics-based approximation to mus- cle fascicle length simplifies Eq. 3.3 to: ~ l m (t) l MT (t) cos( c ) +l m (t o ) (3.6) which, in effect, has led the field to often equate muscle fascicle length changes to MT excursions for small pennation angles (Valero-Cuevas et al., 2009) 6 . Such approximations have greatly facilitated and enabled inferences to be made regarding mus- cle fascicle lengths and velocities or spindle (afferent) activity for multiple tasks (Stanev and Mous- takas, 2019; Hagen and Valero-Cuevas, 2017; Berry et al., 2017; Valero-Cuevas et al., 2015) as well as joint torque vs. angle and joint torque vs. angular velocity studies, where muscle fascicle length/velocity are assumed to be bijective functions of joint angle/angular velocity (Pain et al., 2013; Hahn et al., 2011). Alternatively, the neuromechanical simulation software OpenSim offers a “stiff tendon” mode that makes similar assumptions (Millard et al., 2013). In the aforementioned approaches, pennation angle is generally ignored or assumed constant. This practical approach to MT function has enabled computational studies to infer the neural control strategies of mus- culoskeletal systems (Zajac, 1989; Scott, 2004; Jordan and Wolpert, 1999; Todorov and Jordan, 2002; Valero-Cuevas et al., 2009), even though such models of muscle are known to be prone to parameter sensitivity (Perreault et al., 2003; Scovil and Ronsky, 2006). Here we explore how these approximations and sensitivities affect the conclusions that can be drawn from such muscle models—independently of the assumed contractile element. It is impor- tant to note that our analysis has implications to most lumped-parameter models (e.g., Hill-type models, Hill, 1953) and population-of-muscle-fibers models (e.g., the Fuglevand model, Fugle- 6 While these two assumptions are typically made together, they have been separated here to demonstrate the relative error contribution for which each assumption accounts. 65 vand et al., 1993) because such models tend to ignore tendon mechanics and consider pennation angles to be negligible. We first explicitly derive equations for the errors produced when making a variety of assumptions to enable the reader to make informed decisions about their impact on the particular scientific question or clinical condition studied. We then derive a novel equation to better approximate MT lengths and velocities that considers the effect of changes in moment arm values across postures—something that previous equations fails to capture. And lastly, we will explore the consequences to inter-muscle and inter-subject variability, and its impact on studies of neuromuscular control. 3.3 Material and Methods 3.3.1 Derivation of Tendon Deformation as a Function of Tendon Forces The elastic properties of collagen are such that tendon force (f T ) depends on tendon length (l T ), with a characteristic nonlinear “toe” region at low forces followed by a linear region at higher forces (Ker, 1981; Zajac, 1989; Shadwick, 1990; Brown et al., 1996). To generalize this relationship across muscles, Brown et al. (1996) modelled the normalized tendon force-length curve (Eq. 3.7a) wherebyf T was normalized by the maximum isometric force of the muscle (i.e., ^ f T =f T =f max ) andl T was normalized by the optimal tendon length (i.e., ^ l 0 T =l T =l T;o ). Addition- ally, Brown et al. (1996) used the parametersc T ,k T , andL T r to fit the asymptotic slope, curvature, and lateral-shift, respectively. We will explicitly derive the relationships these parameters have with the tendon force-length curve in Section 3.3.4, but for now it suffices to state thatc T andk T are proportional to the asymptotic slope and radius of curvature in the “toe” region, respectively. Additionally, note that Brown et al. (1996) decided to normalizel T by the optimal tendon length (l T;o — the tendon length when the muscle produces its maximal isometric force) instead of the slack length (l T;s – tendon length when tendon force is negligible) as it produced more “congruent curves.” However, the literature reports the ratio between tendon slack length and optimal muscle 66 fascicle length (Proske and Morgan, 1987; Zajac, 1989), so we preferred to normalize l T by the slack length. The same relationship can be rewritten as Eq. 3.7b for whenl T is normalized byl T;s instead (i.e., ^ l 0 T = (l T;s =l T;o ) ^ l T ). ^ f T (t) =c T k T ln ( exp " ^ l 0 T (t)L T r k T # + 1 ) =c T k T ln 8 < : exp 2 4 l T;s l T;o ^ l T (t)L T r k T 3 5 + 1 9 = ; (3.7a) (3.7b) This relationship, albeit muscle- and subject-specific, can be inverted to provide tendon length as a function of tendon force (Eq. 3.8) 7 . ^ l T (t) = l T;o l T;s k T ln ( exp " ^ f T (t) c T k T # 1 ) +L T r ! (3.8) Therefore, the normalized change in tendon length can be rewritten as a function of both the current and initial forces on the tendon (Eq. 3.9). ^ l T (t) =k T l T;o l T;s ln 8 > > > > < > > > > : exp " ^ f T (t) c T k T # 1 exp " ^ f T (t o ) c T k T # 1 9 > > > > = > > > > ; (3.9) 3.3.2 Error in Fascicle Length Approximations To allow for better comparison across muscles, we define the relative errors in kinematically- approximated muscle fascicle lengths (denoted by) as the differences between Eq. 3.3 and Eqs. 7 Assuming that rate-specific phenomenon like hysteresis, creep, force-relaxation and short-range stiffness are negligible. 67 3.4-3.6, normalized by optimal muscle fascicle length (l m;o ). When only assumingIT , the error is defined by Eq. 3.10. Intuitively, this error will be equal to the tendon length change that was ignored (Eq. 3.9), projected back onto the line of action of the muscle fascicles. IT (t) = l m (t) ~ l IT m (t) . l m;o = l T;s l m;o ^ l T (t) cos((t)) = l T;s l m;o l T;o l T;s k T cos((t)) ln 8 > > > > < > > > > : exp " ^ f T (t) c T k T # 1 exp " ^ f T (t o ) c T k T # 1 9 > > > > = > > > > ; (3.10a) (3.10b) (3.10c) Alternatively, when including tendon length changes but assumingCP the error is defined by Eq. 3.11. CP (t) = l m (t) ~ l CP m (t) . l m;o =C 1 (t); c l MT (t) l T (t) l m;o + C 2 (t);(t o ) ^ l m (t o ) (3.11a) (3.11b) Where C 1 reflects the proportion of l MT l T that was not projected back onto the line of action of the muscle fascicles but instead some other axis given by c (Eq. 3.12) andC 2 represents the proportion of l m (t o ) incorrectly projected back onto the current line of action of the muscle fascicles (Eq. 3.13). C 1 ((t); c ) = cos( c ) cos((t)) cos( c ) cos((t)) (3.12) C 2 ((t);(t o )) = cos((t o )) cos((t)) cos((t)) (3.13) 68 These now allow the relative error in kinematically-approximated muscle fascicle lengths,(t), to be defined as Eq. 3.14. Note that this error accounts for the proportions of l MT andl m (t o ) that were not mapped onto the line of action of the muscle fascicles (CP assumption) as well as the ignored tendon length change (IT assumption). (t) = l m (t) ~ l m (t) . l m;o =C 1 (t); c l MT (t) l m;o ) Error due to incorrectly projecting MT excursion back onto the line of action of the muscle fascicles + C 2 (t);(t o ) ^ l m (t o ) ) Error due to incorrectly projecting initial muscle fascicle length back onto the line of action of the muscle fascicles + IT (t) ) Error due to ignoring tendon length change (3.14a) (3.14b) Finally, correcting for this error provides a more accurate approximation of normalized muscle fascicle length that takes both limb kinematics and tendon tension in account (Eq. 3.15). ^ l m (t) = (Joint Kinematics) Relative MT excursion projected back onto the line of action of the muscle fascicles z }| { 1 cos((t)) l MT (t) l m;o + Initial muscle fascicle length mapped onto the line of action of the muscle fascicles z }| { cos((t o )) cos((t)) ^ l m (t o ) l T;s l m;o l T;o l T;s k T cos((t)) ln 8 > > > > < > > > > : exp " ^ f T (t) c T k T # 1 exp " ^ f T (t o ) c T k T # 1 9 > > > > = > > > > ; | {z } (Tendon Tension) Relative tendon length change projected back onto the line of action of the muscle fascicles (3.15) 69 3.3.3 New and Improved Equations for Musculotendon Length and Velocity When simulating muscle fascicle and tendon behaviors it is paramount to use accurate equa- tions for MT length (l MT ) and velocity (v MT ). Historically, l MT has been calculated either by tracing the MT routing from origin to insertion across postures (Grieve, 1978), or by approximat- ing the MT excursion (s) induced by joint rotations at every joint crossed (Valero-Cuevas, 2016; Kurse et al., 2012; An et al., 1983). The latter approach assumes constant moment arms such that the MT excursion produced by a joint’s rotation from neutral ( o ) would be equal to the arc length of a sector of a circle with radius equal to the moment arm (r, Eq. 3.16). Therefore,l MT is approximated as the sum of some neutral MT length (l MT;o ) and the MT excursions of all joints crossed (Eq. 3.17). Note that the negative sign in the MT excursion equation ensures that for a positive joint rotation from neutral, a MT with a positive moment arm will shorten. sr( o ) (3.16) l MT l MT;o X i r i ( i i;o ) (3.17) Musculotendon velocity (v MT ) is defined as the change in excursion due to joint rotation(s) over time (Eq. 3.18a). For constant moment arms, this is approximated as the linear combination of joint velocities ( _ i ) scaled by their moment arms (Eq. 3.18b). v MT = lim t!0 X i s i t X i lim t!0 r i i t = X i r i _ i (3.18a) (3.18b) However, moment arms are not constant, and the equations forl MT andv MT must account for 70 changes in moment arms with respect to joint angles. As a first approximation, previous work evaluated Eq. 3.18b with posture dependent moment arms (r i ( i ), Eq. 3.19, Hagen and Valero- Cuevas, 2017). v MT X i r i ( i ) _ i (3.19) Integrating Eq. 3.19 with respect to time reveals that this approximation equatess i to the integral of the moment arm function (r i ( i )) across some joint rotation from neutral ( i i;o , Eq. 3.20). l MT l MT;o Z t to X i r i ( i ) _ i dt =l MT;o X i Z i i;o r i ( i )d i (3.20) While Eqs. 3.20 & 3.19 better approximate l MT and v MT , respectively, they fail to capture how moment arm values change with respect to joint angles. To capture this, we propose a new equation forl MT that relies on the definition of arc length in polar coordinates (Eq. 3.21). l MT =l MT;o X i Z i i;o sgn r i ( i ) s (r i ( i )) 2 + @r i @ i 2 d i (3.21) Note that when moment arms are constant, Eq. 3.17 is recovered. The sgn(r ij ( j )) function returns +1 ifr ij ( j ) > 0 and -1 ifr ij ( j ) < 0 to recover the original relationship between joint rotations and excursion changes (i.e., positive joint rotation would shorten a MT with a positive moment arm). From Eq. 3.21 we derive the new equation forv MT as Eq. 3.22b. v MT = X i @ @ i 2 4 Z i i;o sgn r i ( i ) s (r i ( i )) 2 + @r i @ i 2 d i 3 5 d i dt = X i sgn r i ( i ) _ i s (r i ( i )) 2 + @r i @ i 2 (3.22a) (3.22b) The relationship between excursions used in Eq. 3.17 (purple), Eq. 3.20 (orange), & Eq. 3.21 71 r(θ o ) r(θ f ) ∆θ y x Polar Arc Length Equation s = lim ∆θ 0 ∑ j =0 n-1 (r(θ j)sin(∆θ)) 2 + (r(θ j+1)-r(θ j)) 2 = lim ∆θ 0 ∑ j =0 n-1 ( r(θ j+1)-r(θ j) ) 2 ∆θ (r(θ j)) 2 + ∆θ = ∫ θ o θ f ( ∂r ) 2 ∂θ (r(θ)) 2 + dθ Previous MT Equation s ≈ lim ∆θ 0 ∑ j =0 r(θ j)∆θ n-1 Constant Moment Arm Equation s ≈ r(θ o)(θ f - θ o) Lo Figure 3.2: Evolution of MT excursion equations (s) and their differences. The constant moment arm equation (Valero-Cuevas, 2016; Kurse et al., 2012; An et al., 1983) is the simplest approxima- tion but clearly the arc length (dashed purple) does not accurately convey the true MT excursion (black). This was extended in Hagen and Valero-Cuevas (2017) where the true arc length was approximated by integrating the posture-specific moment arm function (orange). Even for suffi- ciently small , this approach does not completely capture the true MT excursion as it ignores the change in moment arm with respect to the joint angle. Correcting for this, we find the true MT excursion from the equation for arc length in polar coordinates (green). As the true MT excursion relies on the Euclidean of the moment arm and its partial derivative, the error between the approx- imation proposed in Hagen and Valero-Cuevas (2017) and the true equation derived here can be bounded by the triangle inequality (see Eq. 3.23). (green) can be explained graphically in Figure 3.2. By exploiting the limit definition of the integral terms for MT excursion, it is easy to show that the magnitude of the error between Eqs. 3.21 & 3.20 (" MT ) for each joint will be bounded by the triangle inequality (Eq. 3.23). " MT = X i 0 @ Z i i;o sgn r i ( i;j ) s (r i ( i )) 2 + @r i @ i 2 d i Z i i;o r i ( i )d i 1 A = X i sgn r i Z i i;o 0 @ s (r i ( i )) 2 + @r i @ i 2 r i ( i ) 1 A d i = X i " MT;i (3.23a) (3.23b) (3.23c) 72 ) j" MT;i j Z i i;o @r i @ i d i (3.24) jr i ( i )r i ( i;o )j (whenr i is monotonic on [ i;o ; i ]) (3.25) Note that the sign of the error will depend on the sign of the moment arm as well as the direction of the joint rotation such that sign of the error will be consistent with the sign of the MT behavior (i.e., positive for lengthening and negative for shortening). Therefore, errors of this type are always underestimates bounded between zero and Eq. 3.24 (the order of which depends on whether the joint rotation induced MT lengthening or shortening, Table 3.1). Lastly, whenr i is either increasing or decreasing only during the joint rotation then this error is simply bounded between zero and the difference between the moment arm function evaluated at the initial and final posture (Eq. 3.25). Table 3.1: When calculating the MT excursionexcursion induced by the rotation of a joint, ignor- ing how much the moment arm (r i ) changes with respect to the joint angle ( i ) will result in an underestimate of how much the MT has either shortened (red) or lengthened (blue). Whether or not a given joint rotation will caused shortening or lengthening depends on the sign of moment arm (r i ( i )) as well as the sign of the change in joint angle ( i Valero-Cuevas, 2016). Therefore, the associated error in the MT excursion (" MT;i ) will have the same sign as the MT excursion and will be bounded by the intersection of this condition and Eq. 3.24 – which both change with the signs ofr i ( i ) and i . i > 0 i < 0 r i ( i )> 0 R i i;o @r i @ i d i " MT;i 0 0" MT;i R i i;o @r i @ i d i r i ( i )< 0 0" MT;i R i i;o @r i @ i d i R i i;o @r i @ i d i " MT;i 0 73 3.3.4 Definingc T andk T The tendon force-length relationship (given by Brown et al. (1996), Eq. 3.7) has fitting con- stantsc T ,k T , andL T r . As described in Brown et al. (1996), these parameters affect the asymptotic slope of the linear region, the curvature of the plot, and the lateral-shift of the relationship, re- spectively. Additionally, by restricting c T and k T to c T k T < 0:20, L T r can be approximated as L T r 1 1=c T with 0:04% error inL T r , allowing for the often-unknown fitting parameter space to be constrained toR 2 . It can be shown from the limit of the slope of the normalized tendon force-length curve (Eq. 3.7) as ^ l T !1 thatc T is proportional to the asymptotic slope (Eq. 3.26). lim ^ l T !1 d ^ f T d ^ l T = lim ^ l T !1 c T l T;s l T;o 2 4 1 + exp 0 @ L T r l T;s l T;o ^ l T k T 1 A 3 5 1 =c T l T;s l T;o (3.26) Another generalization of the tendon force-length relationship is the stress-strain curve where stress is given by the force normalized by the tendon’s physiological cross-sectional area ( = f T =CSA T ) and strain is given by the deformation of the tendon, normalized by the tendon’s slack length (" = (l T l T;s )=l T;s , Herrick et al., 1978; Ker, 1981; Zajac, 1989; Magnusson et al., 2001). This representation is useful for calculating the elastic modulus of tendon (E T , Young’s modulus for tendinous tissue) as the slope of the linear region. E T = " = f T l T;s CSA T l T = f max CSA T ^ f T ^ l T (3.27) As the slope of the linear region of Eq. 3.7 is given byc T (l T;s =l T;o ), we can rewritec T as Eq. 3.28. This equation expands upon the definition ofc T as the “linear stiffness” parameter and provides physical intuition about how changes in physiological parameters like tendon cross sectional area 74 and muscle force producing capabilities will affect it. c T = l T;o l T;s E T CSA T f max (3.28) As the result of this relationship, tendon length changes at high forces can be approximated as ^ l T = 1 c T l T;o l T;s ^ f T = f max ECSA T ^ f T (3.29a) (3.29b) How the fitting parameterk T relates to the curvature of “toe” region of the normalized tendon force-length relationship is not explicitly clear. To explore the effect that changes in these param- eters have on curvature, we define curvature as Eq. 3.30. This definition considers the amount of change that the tangent vector along a curve has in the direction of the normal vector. In the case where the curve is a circle, the curvature is defined as the inverse of the radius. Thus, as the radius decreases, the curvature increases and the “sharpness” of the curve increases. In the case where the radius becomes very large, the curvatures goes to zero as the circle locally approaches a straight line. Therefore, large curvature values () are associated with sharp changes along the curve. In mechanical systems, this is often approximated as the second derivative of the curve. = @ 2 ^ f T . @ ^ l 2 T 1 + @ ^ f T . @ ^ l T 2 3=2 (3.30) = 1 c T k T c T l T;s l T;o 2 exp " L T r l T;s l T;o ^ l T k T # 1 + exp " L T r l T;s l T;o ^ l T k T #! 0 @ 1 + exp " L T r l T;s l T;o ^ l T k T #!2 + c T l T;s l T;o 2 1 A 3=2 (3.31) 75 = C T c T k T z (1 +z) (1 +z) 2 +C T 3=2 (3.32) where,z = exp " L T r l T;s l T;o ^ l T k T # andC T = c T l T;s l T;o 2 ! Ask T is defined as the variable that affects curvature (which varies along the continuous curve), we derive the maximum curvature of the tendon force-length curve in order to see the influence thatk T has on it. To do so, we take the derivative of Eq. 3.30 with respect to ^ l T and find its zeros. @ @ ^ l T = @ @z @z @ ^ l T = C T c T k T l T;s l T;o z 2 4 (1+2z) ( (1+z) 2 +C T) 3=2 3z(1+z) 2 ( (1+z) 2 +C T) 1=2 ((1+z) 2 +C T ) 3 3 5 = C T c T k T l T;s l T;o z (1 +z) 2 +C T 5=2 | {z } 6= 0 h (1z)(1 +z) 2 +C T (1 + 2z) i | {z } max when equal to zero. (3.33a) (3.33b) (3.33c) Therefore the curvature is at a maximum when (1z)(1 +z) 2 +C T (1 + 2z) = 0 which will occur when, z = 3 p 2(6C T + 4) 3 3 q 9C T + 16 +j3 p 3 p 32C 3 T + 61C 2 T + 32C T (3.34) + 3 q 9C T + 16 +j3 p 3 p 32C 3 T + 61C 2 T + 32C T 3 3 p 2 1 3 Representing Eq. 3.34 in terms of magnitude and phase allows us to remove the imaginary com- 76 ponent. z = 3 p 2(6C T + 4) 3 q 9C T + 16j3 p 3 p 32C 3 T + 61C 2 T + 32C T 3 3 p (9C T + 16) 2 + 27(32C 3 T + 61C 2 T + 32C T ) + 1 3 3 p 2 3 r 9C T + 16 +j3 p 3 q 32C 3 T + 61C 2 T + 32C T 1 3 = 1 3 3 p 2 p 4(6C T + 4) 3 e j 1=3 + 1 3 3 p 2 p 4(6C T + 4) 3 e j 1=3 1 3 = 2 3 p 6C T + 4 cos 3 1 3 r 6C T + 4 3 1 3 p 2C T =c T l T;s l T;o p 2 where = tan 1 s 27(32C 3 T + 61C 2 T + 32C T ) (9C T + 16) 2 ! =2 ! (3.35a) (3.35b) (3.35c) (3.35d) (3.35e) Therefore the maximum curvature is given by plugging the Eq. 3.35c into Eq. 3.32. max = z = C T c T k T z (1 +z ) (1 +z ) 2 +C T 3=2 (3.36a) (3.36b) The approximate maximum curvature is found by instead evaluating the curvature at the approxi- mate value forz (Eq. 3.35e). Consideringc T >> 10 (see Section 3.3.5), we can manipulate the 77 equation for maximal to find, max C T c T k T p 2C T 1 + p 2C T 1 + p 2C T 2 +C T 3=2 = C T c T k T 2C T + p 2C T 3C T + 2 p 2C T + 1 3=2 = C T c T k T 2 p C T + p 2 4 2 1 4 3 3=2 p C T + p 2 3 2 + 1 9 3=2 C T c T k T 2 3 p 3 1 p C T + p 2 3 p C T c T k T 2 3 p 3 ) max 2 p 3 9 ! l T;s l T;o 1 k T (3.37a) (3.37b) (3.37c) (3.37d) (3.37e) (3.37f) (max) 1 = minR 3 p 3 2 ! l T;o l T;s k T (3.38) Therefore, as Eq. 3.37f suggests, the value ofk T is inversely proportional to the curvature and therefore proportional to the radius of curvature of the “toe” region (R), where small values ofk T correspond to high curvature or low radius of curvature (i.e., would exhibit sharp transitions from the “toe” region to the “linear” region). Additionally, Eq. 3.37f and its reciprocal, Eq. 3.38, could be used to help find the hard-to-measure k T constant from an experimental, normalized tendon force-length curve by either calculating the curvature and finding its maximum or by measuring the smallest radius of curvature, respectively. 78 3.3.5 Defining Physiological Ranges forc T &k T The parameters that are used to characterize the normalized tendon force-length curve—asymptotic stiffness, the radius of curvature constant, and lateral shift (c T ,k T , andL T r , respectively)—greatly influence the behavior of the tendon and vary across muscles, subjects, and sometimes tension rates (Maganaris and Paul, 2000; Finni et al., 2013). In order to determine the effects that chang- ing these fitting parameters have on overall MT behavior, we define the physiological range forc T andk T from (i) the condition thatc T k T < 0:20 (Brown et al., 1996), and (ii) the condition that, by defintion, the tendon must be at its slack length ( ^ l T = 1) when normalized tendon force ( ^ f T ) is near zero (Proske and Morgan, 1987; Zajac, 1989; Brown et al., 1996; Magnusson et al., 2001). The first constraint onc T andk T comes from Brown et al. (1996), who stated that restricting val- ues toc T k T < 0:20 allowsL T r in Eq. 3.7 to be approximated asL T r 1 1=cT while incurring only a 0.04% error inL T r (red region excluded in Figure 3.3). The second constraint produces a range of acceptable values ofc T andk T when satisfying Eq. 3.8 for ^ f T 0 and ^ l T = 1 and for c T k T < 0.20 f T ≈ 0 → l T = l T,s f T ≈ 1 → l T = l T,o ^ ^ 20 40 60 c T ~ Linear Stiffness 0.006 0.004 0.002 0 k T ~ Radius of Curvature Physiologically Realistic Values of c T and k T Figure 3.3: Physiologically realistic ranges forc T andk T under the assumptions that (i)c T k T < 0:20 (red, Brown et al., 1996), and (ii), by definition, when force is negligible, tendon length equals its slack length (i.e., ^ f T 0! ^ l T 1, blue, Proske and Morgan, 1987; Zajac, 1989; Brown et al., 1996; Magnusson et al., 2001). 79 l T;s =l T;o 2 [1:03; 1:0:7] (blue regions excluded in Figure 3.3, Magnusson et al., 2001; Maganaris and Paul, 2002). To validate the ranges produced by these constraints, we explore the reported ranges for c T as (i) we can calculatedc T from Young’s modulus (E), the tendon’s physiological cross-sectional area (CSA T ), and the maximum isometric force of the muscle (f max ) from Eq. 3.28 and (ii) the values of k T are less often reported. Young’s modulus has been reported to be conserved across muscles with the average E reported to be around 1.2 GPa (Bennett et al., 1986; Zajac, 1989; Magnusson et al., 2001; Pollock and Shadwick, 2017) and, therefore, changes toc T can be attributed to changes inCSA T andf max . These two parameters change across MTs as well as with training, injury, or pathology and help to explain the large variability in tendon stiffness seen across muscle and subjects (Hof, 1998; Maganaris and Paul, 2002; Lichtwark and Wilson, 2005). As an example, Magnusson et al. (2001) calculated E, CSA T , and f max for the medial gastrocnemius of 5 individuals during isometric contraction tasks and the c T values calculated from Eq. 3.28 ranged from 23.23 to 65.70 (37:47 14:88). Therefore, the range ofc T values produced by the two constraints described above are consistent with values reported in the literature and are good first approximations of the range of physiological values when exploring the affect they have on tension-specific tendon deformation. 3.4 Results 3.4.1 Error from Ignoring Fascicle Pennation As the first two terms of Eq. 3.14b illustrate, assuming that pennation angle is either constant or negligible will result in errors when approximating muscle fascicle lengths kinematically (Ma- ganaris et al., 1998). The coefficient of the first term (C 1 ) reflects the percentage of l MT that was not projected back onto the line of action of the muscle fascicles, but instead some other axis given 80 40° 30° 20° 10° 0° Actual Pennation Angle ( ρ) 40° 30° 20° 10° 0° Assumed Constant Pennation Angle ( ρ c ) -20% -10% Zero Error 10% 20% 20% 0% -20% Percent ∆l MT (t) Error l m,o 40° 30° 20° 10° Higher ρ c 20% 0% Percent ∆l MT (t) Error l m,o -20% 40° 30° 20° 10° Lower ρ c Percent of Musculotendon Excursion Not Projected onto the Line of Action of the Fascicles as the Result of Assuming Constant Pennation Figure 3.4: Contour map (left) for the percentage of MT excursion that would be incorrectly mapped onto the line of action of the muscle fascicles (C 1 ) as the result of assuming some constant pennation angle as a function of the true pennation angle. For any assumed value of constant pen- nation angle ( c ), the resulting plot ofC 1 is given by the corresponding horizontal cross-section of the contour plot. Examples of plots for lower and higher values of c are shown on the right. Note that the error is negligible when the true pennation angle is equal to the assumed value (diagonal line on the left, and zero-crossings on the right). Additionally, the error is less than5% when the assumed and actual pennation angles are less that 18 . 81 A B ρ c = 0° 10° 20° 30° 40° 10° 20° 30° 40° 10% -10% 5% 0% -5% Percent of ∆l MT Not Projected onto the Line of Action of the Fascicles Pennation Angle Soleus Triceps Bracchi Wrist & Finger Muscles Vastus Lateralis Medial Gastrocnemius Tibialis Anterior ρ(t o ) = 0° 10° 20° 30° 40° 10° 20° 30° 40° 8% -8% 4% 0% -4% Percent of l m (t o ) Not Projected onto the Line of Action of the Fascicles Pennation Angle Figure 3.5: Sensitivity of the relative error coefficients C 1 (the proportion of MT excursion not projected back onto the line of action of the muscle fascicles, A) and C 2 (the proportion of the initial muscle fascicle length not projected back onto the line of action of the muscle fascicles, B) from Eq. 3.14. A small deviation (5 ) was applied to c or(t o ), respectively, and the resulting change in the coefficients were plotted. For (A), while the error is minimized when(t) = c , as c increases the same deviation from the true pennation angle ((t) = c 5 ) will produce larger changes inC 1 and, therefore, a larger percentage of l MT would be incorrectly projected back onto the line of action of the muscle fascicles. Similarly for (B), the error will be minimized when the pennation angle does not change from the initial value (i.e.,(t) = (t o )), but as(t o ) increases, the same deviation from the initial value ((t) =(t o )5 ) will result in larger changes toC 2 and, therefore, a larger proportions of the initial muscle fascicle length would be incorrectly mapped back onto the muscle fascicles at timet. Ranges of pennation angles reported in the literature have been provide for a few muscle groups for reference (Yamaguchi et al., 1990; Lieber et al., 1990; Lieber and Brown, 1992; Herbert and Gandevia, 1995; Fukunaga et al., 1997; Martin et al., 2001; Ward et al., 2009; Kwah et al., 2013). 82 by c (Eq. 3.12, Figure 3.4). We define the sensitivity of this coefficient to be the partial derivative with respect to evaluated at = c as this describes how much the coefficient would change for pennation angle changes near the assumed constant value. @C 1 @ =c = tan( c ) sec( c ) (3.39) And while its obvious that errors of this type will be negligible when (t) c , it becomes more sensitive to small changes in pennation angle as c increases (i.e., tan( c ) sec( c )!1 as c !=2; Figure 3.5A). Therefore, when assuming CP for muscles with larger pennation angles, the same deviation ( = c 1 ) will result in a larger percentage of l MT that was incorrectly projected back onto the line of action of the muscle fascicles. The coefficient of the second term of Eq. 3.14b (C 2 ) reflects the percentage ofl m (t o ) incorrectly projected back onto the line of action of the muscle fascicles as the result of assuming constant pennation angle (Figure 3.6). Similar toC 1 , we define the sensitivity ofC 2 as the partial derivative with respect to evaluated at = (t o ) to capture the effect of changes in pennation angle near the initial pennation angle. @C 2 @ =(to) = tan((t o )) (3.40) Errors of this type will likewise be negligible when the pennation angle does not deviate from the initial value ((t)(t o )), but the sensitivity to changes in pennation angle increases as(t o ) increases (i.e., tan(t o )!1 as(t o )! =2; Figure 3.5B). Therefore, when assuming CP for muscles with larger pennation angles that also undergo larger pennation angle changes, the same deviation ((t) = (t o ) 2 ) will result in a larger percentage ofl m (t o ) that was not projected back onto line of action of the muscle fascicles. Reported average pennation angles vary across muscles in humans; from 1:3 to 46:1 83 Pennation Angle ( ρ) 40° 30° 20° 10° 0° 40° 30° 20° 10° 0° Initial Pennation Angle ( ρ(t o )) -20% -10% Zero Error 10% 20% 20% 0% -20% Percent l m (t o ) Error ^ 40° 30° 20° 10° Higher ρ(t o ) Percent of Initial Muscle Length Not Projected onto the Line of Action of the Fascicles as the Result of Assuming Constant Pennation 20% 0% -20% Percent l m (t o ) Error ^ 40° 30° 20° 10° Lower ρ(t o ) Figure 3.6: Contour map (left) for the percentage of initial muscle fascicle length that would be incorrectly mapped back onto the true line of action of the muscle fascicles (C 2 ) as the result of assuming some constant pennation angle as a function of the initial and current pennation angles. Regardless of the assumed constant pennation angle value, this error will depend on the amount by which the pennation angle changes from its initial value. Therefore, for some initial pennation angle, the resulting plot of the coefficientC 2 is given by the corresponding horizontal cross-section of the contour plot. Examples of plots for lower and higher values of(t o ) are shown on the right. Note that the error is negligible when the current pennation angle is equal to the initial pennation angle (diagonal line on the left, and zero-crossings on the right). Additionally, the error is less than 5% when the initial and actual pennation angles are less that 18 . Therefore, for muscles with small pennation angles (< 18 ) that do not drastically change over the course of a movement, this error will be relatively small (<5%). 84 in the lower limb (Ward et al., 2009; Fukunaga et al., 1997; Yamaguchi et al., 1990; Martin et al., 2001) and from 2:0 to 33:0 in the upper limb (Lieber et al., 1990; Lieber and Brown, 1992; Herbert and Gandevia, 1995; Kwah et al., 2013) with large variability reported across sub- jects. Additionally, pennation angles have been reported to change by 120% to 175% from rest when the muscle is fully activated (Herbert and Gandevia, 1995; Narici et al., 1996a,b; Kawakami et al., 1998; Maganaris and Baltzopoulos, 1999; Kurokawa et al., 2001). Therefore, for some mus- cles and movements, the percentage of either l MT orl m (t o ) that can be excluded or incorrectly mapped back onto the line of action of the muscle fascicles can be quite large (20%). As an ex- ample, Kurokawa et al. (2001) reported that the medial gastrocnemius experienced a l MT 2 cm during a maximum squat jump, where the pennation angle increased from 20 to 35 . If the pennation angle had been assumed to be constant and equal to the initial pennation angle (i.e., c =(t o ) = 20 ), then 15.7% of l MT and 14.7% ofl m (t o ) would have been incorrectly mapped back onto the muscle fascicle, resulting in an normalized error of around -6.5% (3:13mm) and 17.2% (8:24mm), respectively, givenl m (t o ) = 5:6 cm andl m;o = 4:8 cm (Kurokawa et al., 2001). 3.4.2 Error from Assuming Inextensible Tendon The third term of Eq. 3.14b represents the error associated with assuming inextensible tendons (Hoffer et al., 1989). Figure 3.7 shows Eq. 3.14b evaluated at two initial tendon tensions. Note that when ^ f T = ^ f T (t o ) (i.e., the normalized tendon force intercept), the tendon will have undergone a net zero length change and the error will be zero. However, notice that the slope at the intercept (red lines) demonstrates that starting at lower forces (i.e., an intercept towards the left) creates greater sensitivity to deviations from the initial tension, whereas at higher forces the sensitivity is lower and approaches the asymptotic slope (black arrows). Therefore, tasks that simulate non-isotonic tendon forces at low levels (like most activities of daily living) are at the greatest risk of errors of this type. However, the parameters used to characterize the shape of the curves in Figure 3.7 (i.e., c T , k T , l T;o =l T;s , and l T;s =l m;o ) vary across muscles and subjects and can change with exercise, 85 f T (t o ) ^ 2 f T (t o ) ^ 1 1 l T,o l T,s l T (to) - ^ ) ( l T,s l m,o cos( ρ) 1 l T,o l T,s l T (to) - ^ ) ( l T,s l m,o cos( ρ) 2 slope ≈ - l T,s l m,o c T cos(ρ) ( 1 - exp [ - fT(to) / c T k T ] ) -1 ^ i l T,o l T,s ) ( ) ( asymptotic slope ≈ - l T,s l m,o c T cos( ρ) 1 l T,o l T,s ) ( ) ( Normalized Tendon Force Relative Error in Normalized Fascicle Length Figure 3.7: Relative error in muscle fascicle length as a result of assuming inextensible tendons (Eq. 3.10c) that accounts for the previously ignored tendon length change, scaled by the tendon slack length to optimal muscle fascicle length ratio (l T;s =l m;o ), and projected back onto the line of action of the muscle fascicles. Note that the error will be zero when the tension of the tendon is equal to the initial tension ( ^ f i T (t o )—i.e., no net deformation of the tendon has occurred). Two different initial tension values have been chosen to demonstrate that starting at lower forces (i.e., an intercept towards the left) creates greater sensitivity to deviations from the initial tension, whereas at higher forces the sensitivity is lower and approaches the asymptotic slope (black arrows) where it will be proportional to l T;s =l m;o and inversely proportional to c T = ECSA T =f max (i.e., the tendon’s normalized asymptotic stiffness). . injury, or pathology (Lichtwark and Wilson, 2007, 2008; Arampatzis et al., 2006; Muraoka et al., 2005; Roy and Edgerton, 1992; Hof et al., 2002; Magnusson et al., 2001; Maganaris and Paul, 2002). Additionally, changes across muscles and subjects, as well as under different force levels for a given muscle (as previously stated) and will affect the error proportionally. Therefore, we explored how changes in these parameters affect the magnitude of the error in muscle fascicle lengths by simulating isometric force tasks from rest. By doing so, MT excursion will be zero such that kinematically-approximated muscle fascicle lengths will be constant and the subsequent error will be identical to the normalized length change of the tendon, scaled byl T;s =l m;o and divided by the cosine of the pennation angle. Figure 3.8A shows 10,000 random isometric force tasks (between 0-100% of maximum voluntary contraction, MVC) with the 5 parameters of 86 (A) -60% -40% -20% 0 -80% 20 30 40 50 60 0 0.001 0.002 0.003 0.004 0.005 0.006 1.03 1.04 1.05 1.06 1.07 0 2 4 6 8 10 11.25 0° 10° 20° 30° 40° 0 20% 40% 60% 80% 100% 0 0.001 0.002 0.003 0.004 0.005 0.006 1.03 1.04 1.05 1.06 1.07 0 2 4 6 8 10 11.25 0° 10° 20° 30° 40° 0 20% 40% 60% 80% 100% 20 30 40 50 60 -60% -40% -20% 0 -80% (D) Low Error (≤ 5%) • No Dependence on Force Level or Musculotendon Architecture 0 0.001 0.002 0.003 0.004 0.005 0.006 1.03 1.04 1.05 1.06 1.07 0 2 4 6 8 10 11.25 0° 10° 20° 30° 40° 0 20% 40% 60% 80% 100% 20 30 40 50 60 -60% -40% -20% 0 -80% (B) • No Dependence on c T or k T • Proportional to and High Error / Higher Forces (≥ 30%) / (≥ 75%) 0 0.001 0.002 0.003 0.004 0.005 0.006 1.03 1.04 1.05 1.06 1.07 0 2 4 6 8 10 11.25 0° 10° 20° 30° 40° 0 20% 40% 60% 80% 100% 20 30 40 50 60 -60% -40% -20% 0 -80% (C) High Error / Lower Forces (≥ 30%) / (≤ 50%) • “Perfect Storm” of Parameters • Smaller Slope in “Toe” Region c T (~Linear Stiffness) k T (~Curvature -1 ) MVC η (Error in l m,o ) ρ (Pennation Angle) 87 Figure 3.8: Parallel coordinates plot for the relative error in muscle fascicle length () associated with assuming inextensible tendons for 10,000 random samples in the 5 parameters of interest (c T , k T , l T;o =l T;s , l T;s =l m;o , and ) within their reported physiological ranges during random isomet- ric force tasks (top). As MT excursion (l MT ) is zero during isometric contractions, assuming inextensible tendons is equivalent to assuming that muscle fascicle length is constant at a given percentage of maximum voluntary contraction (MVC) and the error, therefore, will be identical to the normalized length change of the tendon, scaled byl T;s =l m;o andl T;o =l T;s , and divided by the cosine of the pennation angle. For higher forces ( 75% MVC), we find that high errors ( 30% l m;o ) can occur for all values of c T and k T as, by definition, the tendon length converges to its optimal length (i.e., the error is only proportional tol T;o =l T;s andl T;s =l m;o , second from top). Al- ternatively, we find that the error can be equally large for lower forces ( 50% MVC) when the tendon has low stiffness, low curvature (high radius of curvature), and larger ratios ofl T;s tol m;o andl T;o =l T;s (i.e., the “perfect storm”, second from bottom). Conversely, if any of these conditions are not met, the errors in muscle fascicle lengths can be low (bottom). Lastly, pennation angles do not appear to preclude any muscles from this sort of error, but it is trivial to show that increasing will increase the proportion of l T projected back onto the line of action of the muscle fasci- cles. Visit https://daniel8hagen.com/images/tendon length change parallel coords to access this interactive parallel coordinate plot online. interest uniformly sampled within their reported physiological ranges (Santos et al., 2009) 8 . Based on the reported range for l T;o =l T;s (and the definitions of l T;s and l T;o ) the maximal normalized tendon deformation we can expect at MVC will be max MVC ^ l T = l T;o l T;s l T;s = l T;o l T;s 12 [0:03; 0:07] such that the maximal error in muscle fascicle length for this task will be the maximal tendon de- formation scaled by the ratiol T;s =l m;o and divided by the cosine of the pennation angle for a given muscle. For physiological ranges of these values, the maximal error in muscle fascicles can reach magnitudes of80%l m;o . As expected, for higher tendon forces ( 75% of MVC) where ^ l T con- verges tol T;o =l T;s , those trials with larger errors ( 30%l m;o ) shows no clear dependence onc T or k T , but instead are determined by the product ofl T;o =l T;s andl T;s =l m;o (Figure 3.8B). Surprisingly, for lower tendon forces ( 50% of MVC), errors in muscle fascicle lengths 30% (Figure 3.8C) 8 The range for was chosen to be< 40 (See Section 3.4.1), the range forl T;o =l T;s was chosen to be 1.03-1.07 (Cheng et al., 2000; Magnusson et al., 2001; Maganaris and Paul, 2002), and the range forl T;s =l m;o has been reported to be 11:25 (Zajac, 1989; Hoy et al., 1990). For an explanation of the physiological levels ofc T andk T see Section 3.3.5. 88 typically occur for MTs with lower normalized asymptotic tendon stiffness (lowerc T ), lower “toe” region curvature in the tendon’s force-length curve (higherk T ), and higher ratios of tendon slack length to optimal muscle fascicle length (l T;s =l m;o > 6) and optimal tendon length to tendon slack length (l T;o =l T;s > 1:04). These parameters corresponds to a tendon that (i) is substantially longer than the muscle (i.e., largerl T;s =l m;o ), (ii) can undergo relatively larger overall deformations (i.e., largerl T;o =l T;s ), and (iii) has a flatter “toe” region in the tendon force-length curve. This “perfect storm” elicits disproportionately greater tendon deformations per unit force at lower forces. Con- versely, if any of these conditions are not met, the errors in muscle fascicle lengths can be low ( 5%l m;o , Figure 3.8A). Lastly, it is of course trivial to show from Eqs. 3.2 & 3.3 that increasing the pennation angle will increase the proportion of l T projected back onto the line of action of the muscle fascicles (i.e., a 1= cos((t)) function), and thus increase the magnitude of this error. However, we see that even muscles with low pennation angles are not immune to high errors (see second column from right in Figure 3.8B,C). To further explore the consequences of varying these parameters during an isometric task, the reader is pointed to the online interactive version of Figure 3.8, available at https://daniel8hagen.com/images/tendon length change parallel coords. 3.5 Discussion Biomechanics and neuromuscular control depend uniquely on the assumed properties of mus- cle. Chief among them is the dependence of muscle force for a given amount of neural drive on the lengths and velocities of its muscle fascicles. Our detailed analysis of how muscle fascicle lengths (and, by extension, velocities) are approximated from experimental data or simulations points to assumptions regarding MT architecture (i.e., pennation angle and tendon elasticity) as the source of three dominant types of uncertainty and error (Eq. 3.14b). More importantly, the magnitude of these errors is highly sensitive to the parameters characterizing MT architecture. The significance of these potentially large errors is ultimately left to the modeller and their audience. To guide these decisions in practice, however, we provide a detailed presentation of the interactions between MT 89 architecture, MT force magnitude, as well as its dependence on the ratios of tendon slack length to optimal muscle fascicle length and of optimal tendon length to tendon slack length. These high- dimensional interactions show that some combinations of parameters will produce (i) relatively low errors vs. (ii) a “perfect storm” where errors will be unacceptably large by any measure, even for relatively low force magnitudes. Our work highlights that extreme care must be taken when making such approximations in the context of the scientific question being asked, the muscles and tasks being studied, and the available experimental data. We began by showing that, when approximating muscle fascicle lengths from the kinemat- ics, separately assuming either constant pennation angle or inextensible tendons will incur distinct errors with respect to the ground truth. The magnitude of these errors are themselves highly sen- sitive to the parameters that characterize the MT (i.e., muscle fascicle pennation angle and tendon elasticity). It is intuitively obvious that assuming zero or constant pennation angle will incorrectly map the changes in MT length and the initial muscle fascicle length onto the current line of action of the muscle fascicles. Our community has often assumed that this error is negligible for “small” pennation angles, which may be true (Figure 3.5A). However, we show that starting at reasonably modest pennation angles of 20 , the sensitivity of the errors escalates exponentially to the point where a5 deviation from the assumed constant pennation angle can lead to> 5% of the MT excursion to be unaccounted for in the muscle fascicle length approximation—with potentially important consequences to force-length and force-velocity calculations. Similarly, it is known that some muscles with modest pennation angles nevertheless undergo large changes in pennation angle during everyday movements (c. 120–175%, Herbert and Gandevia, 1995; Narici et al., 1996a,b; Kawakami et al., 1998; Maganaris and Baltzopoulos, 1999; Kurokawa et al., 2001). In these cases, the amount of initial muscle fascicle length that is incorrectly mapped back onto the line of action of the muscle fascicles can be>> 4% as calculated for a5 change in pennation angle (Figure 3.5B) and can reach magnitudes on the order of 20% (see Kurokawa et al. (2001) example in 90 Section 3.4.1). Both of these errors increase greatly for more pennate (and highly studied) muscles such as tibialis anterior, triceps bracchi, medial gastrocnemius, or soleus. This highlights the need for perhaps unrealistically or impractically accurate measurements of muscle pennation angle (and their change during a task) to accurately simulate force production in these muscles when they are operating away from the plateau of the force-length curve, and most everywhere in the force- velocity curve. Additionally, if one wanted to assume inextensible tendons, one must note that the errors in estimated muscle fascicle lengths are exacerbated at lower forces ( 50% of MVC) if the com- bination of parameters that characterize the tendon’s normalized force-length relationship (i.e., its asymptotic stiffness,c T , and its curvature constant,k T ) and the ratios of the tendon’s slack length to the optimal muscle fascicle length (l T;s =l m;o ) and optimal tendon length to tendon slack length (l T;o =l T;s ) meet the “perfect storm” criteria (e.g., soleus muscle, Magnusson et al., 2001; Zajac, 1989; Hoy et al., 1990). That is, errors can be quite large for more compliant tendons (i.e., low values ofc T and high values ofk T ) with moderately high values ofl T;s =l m;o andl T;o =l T;s (Figure 3.8C). Conversely, if any of these “perfect storm” conditions are not met, the error can poten- tially be small, as can be seen by the wide distribution of parameters that produce less than 5% error in muscle fascicle lengths for the isometric force task illustrated in Figure 3.8D. Therefore, it is important to understand how a particular choice of parameters can affect the robustness of a kinematics-based muscle fascicle length approximation by understanding how they will effect the shape of the tendon’s force-length relationship and how deformations of tendon will subsequently produce scaled changes in muscle fascicle length. The limitations of this study do not necessarily affect the validity of our results. For example, we did not consider more complex MT architectures that do not reflect the simple parallelogram, and we ignored hysteresis, tendon creep, short-range stiffness, and force-relaxation. In fact, the added nonlinearities and state-dependence of these omissions suggest that we may, in fact, be un- derestimating these uncertainties and errors. By correcting for the errors explicitly derived here, we 91 arrive at a more accurate approximation of muscle fascicle length that relies on both limb kinemat- ics and tendon dynamics (Eq. 3.15). This equation creates a functional relationship, through the limb kinematics, between muscle fascicle lengths and tendon tensions for concentric, eccentric and isometric contractions. However, in practice, its sensitivity to parameter values makes it all-but- impossible to use to recover specific MT behavior without accurate subject- and muscle-specific parameters. Another apparent limitation to this study may be the exclusion of muscle activations in the determination of muscle fascicle lengths and velocities. It is well established that muscle mechan- ics can be approximated by a second order dynamical system, where muscle activation produces state-dependent muscle force to change the length and velocity of the muscle fascicles, ultimately pulling on the tendon to actuate the joints. As discussed in Chapter 2, careful consideration must be made when designing a controller with muscle activations as the inputs, and changes in inputs can change the overall behavior of the muscle fascicles. But these activation changes likewise change the tendon tensions that result from such activations, preserving the relationship between tendon tension, muscle fascicle lengths, and the resulting kinematics (if it exists). So while this work excluded the explicit use of so called Hill-type models in our discussion, it does not preclude the use of our purely descriptive model which captures this functional relationship between mus- cle and MT through the tension-specific deformation of tendon. More specifically, the results from this chapter are built upon the assumptions that (i) MT excursions are fully determined by the joint kinematics and (ii) muscle fascicle and tendon lengths must always sum to MT length (thereby accounting for the rotation/excursion that the tendon forces ultimately induced). Ultimately, any simulation study that utilizes Hill-type muscle models will encounter similar errors if muscle fas- cicle lengths are not updated by this relationship (i.e., the role of tendon deformation or pennation angle in MT excursion are ignored). These results also raise further important issues relevant to the role of muscle spindles to pro- vide proprioception for neuromuscular control. In Nature, proprioception provides animals a ro- 92 bust awareness of the state of their body and of their relation to the environment. The muscle spindles embedded in the intrafusal fibers of muscle are known to provide information about the velocity and length of a muscle. Muscle spindles, by sensing along the line of action of the muscle fascicles, are nevertheless subject to similar uncertainties and errors to those described above when estimating the length of the MT. However, it stands to reason that the nervous system can learn to interpret and integrate spindle signals to estimate the posture of the limb because their afferent signals are implicitly functions of the subject- and muscle-specific MT parameters. Nevertheless, Eq. 3.15 points to the physical necessity of knowing tendon tension in addition to muscle fascicle length to accurately estimate the length (and velocity) of the MT—and therefore limb posture. We speculate that this obligatory functional relationship between muscle fascicle lengths and tendon tensions may point to an additional evolutionary pressure for Golgi tendon organs (mechanorecep- tors for tendon tension) whose projection to the same spinal, sub-cortical and cortical areas would critically enable more accurate estimates of limb posture. And if this relationship exists in Nature, could we use this information to our advantage for robotic design? More specifically, can we design an algorithm that predict joint angles from non- collocated sensory signals in a compliant tendon-driven robot? 93 Chapter 4 Measuring tendon tension allows limited-experience neural networks to better predict posture in tendon-driven systems Previous chapters have explored the effect that compliant tendons have on the control of redun- dant, antagonistically-actuated tendon-driven systems and the relationship between the muscles, musculotendons, and, by extension, joint kinematics. We have shown that it is impossible to esti- mate muscle behavior from joint kinematics alone and that tendon tension can be used to reconcile this difference if the muscle- and subject-specific musculotendon parameters are known. While this parametric uncertainty presents a difficult (if not impossible) hurdle for computational mod- elling of subject-specific neuromuscular control, the mere existence of a functional relationship between muscle mechanics, tendon tensions, and joint kinematics for any given set of (unknown) parameters enables an interesting line of questioning and provides the foundation for the final two chapters of this dissertation. Mainly, could we use the knowledge that such functional relationships exist in compliant tendon-driven systems to design an artificial neural network (ANN) algorithm that utilizes information from actuators and tendon tensions to predict joint angles? And if so, what does that say about how the nervous system might integrate sensory information for the internal modeling of posture? The importance of the observability of tendon tensions and actuator states has been emphasized in biology as there are sensors that encode tendon tension (Golgi tendon organs) as well as muscle fascicle lengths and velocities (spindle secondary and primary afferents, respectively) whose spinal and supraspinal projections may integrate to form internal representations of expected or virtual limb position. In tendon-driven robotics, this implies that it may be possible to infer joint angles from motor angles and tendon tensions, thus potentially removing the need for on-location joint encoders (and their contribution to joint inertia, design and construction complications, noise, etc.). 94 Therefore it is both useful and necessary to understand if the relationships between kinematic states, actuator states, and/or tendon tensions can be learned to improve our understanding of biological proprioception in biology, and improve the design and control of bio-inspired, tendon- driven robots. In this chapter, we begin by considering a simple tendon-driven robotic model and exploring how ANNs trained with different amounts of non-collocated sensory information (like tendon ten- sion or motor position) perform when predicting joint angle. The goal of this experiment will be to identify a minimal set of sensory information needed to provide a good estimate of joint angle and if tendon tension information enables more accurate estimates. The final chapter will then address how changes in mechanical properties of the plant (i.e., the tendon’s stiffness or the motor’s damp- ing) affect our findings and whether changes in the dynamic task requirements change the utility of these sensory data in these algorithms. 4.1 Abstract Estimates of limb posture are critical for the control of robotic systems. This is generally accomplished by utilizing on-location joint angle encoders which may complicate the design, in- crease limb inertia, and add noise to the system. Conversely, some innovative or smaller robotic morphologies can benefit from non-collocated sensors when encoder size becomes prohibitively larger or the joints are less accessible or subject to damage (e.g., distal joints of a robotic hand or foot sensors subject to repeated impact). These concerns are especially important for tendon-driven systems where motors (and their sensors) are not placed at the joints. Here we create a framework for joint angle estimation by which artificial neural networks (ANNs) use limited-experience from motor babbling to predict joint angles. We draw inspiration from Nature where (i) muscles and tendons have mechanoreceptors, (ii) there are no dedicated joint-angle sensors, and (iii) dedicated neural networks may perform sensory fusion. We simulated an inverted pendulum driven by an 95 agonist-antagonist pair of motors that pull on tendons with nonlinear elasticity. We then compared the contributions of different sets of non-collocated sensory information (like motor positions or tendon tensions) when training ANNs to predict joint angle. By comparing performance across different movement tasks we were able to determine how well each ANN (trained on the differ- ent sensory sets of babbling data) generalizes to unlearned tasks (sinusoidal and point-to-point). Lastly, we evaluated performance both as a function of motor babbling duration and the number of hidden layer nodes used in each network. We find that training an ANN with actuator states (i.e., motor positions/velocities/accelerations) as well as tendon tension data produces more accurate es- timates of joint angles than those ANNs trained without tendon tension data. Moreover, we show that ANNs trained on motor positions/velocities and tendon tensions (i.e., the Bio-Inspired Set) (i) can reliably estimate joint angles with as little as 15 seconds of motor babbling that (ii) generalize well across tasks. We demonstrate a novel framework that can utilize limited-experience to provide accurate and efficient joint angle estimation during dynamic tasks using non-collocated actuator and tendon tension measurements. This enables novel designs of versatile and data-efficient robots that do not require on-location joint angle sensors. 4.2 Introduction Tendon-driven robots are becoming popular due to a number of advantages these designs can provide (Valero-Cuevas, 2016; Andrychowicz et al., 2019; Marjaninejad et al., 2019b). Elastic tendons can increase energy efficiency by storing potential energy and can protect actuators from impacts by dissipating energy upon impact (Laurin-Kovitz et al., 1991; Pratt and Williamson, 1995; Pratt, 2002; Mazumdar et al., 2017). Additionally, tendon routings offer flexibility to how torques and angular velocities at the motors are converted to torques and angular velocities at the joints (Lee and Tsai, 1991; Kobayashi et al., 1998; Marjaninejad and Valero-Cuevas, 2019; Marjaninejad et al., 2019a). Most importantly, tendon-driven systems offer flexible placement options for the actuators, which eliminate the need for motors to be placed on the joints themselves. Proximal 96 actuator placement moves the center of mass towards the body of the robot thereby reducing limb inertia and allowing for more efficient displacement in quadrupeds or anthropomorphic robots (Jacobsen et al., 1986). However, most successful state-based robotic control strategies need to observe or approximate joint angles which is generally done by placing sensors on the joints (in the absence of alternatives such as visual feedback; Marjaninejad et al., 2019b,c). Although sensors in general have lighter mass than motors, this can still add unwanted inertia to the limbs. These on-location sensors are prone to motion noise and their wiring is often cumbersome and poses a potential risk of damage. These adverse effects become more pronounced for smaller, distal joints where the mechanical design may make the joint inaccessible (e.g., in the case of a tendon-driven finger in a robotic hand). One alternative solution, which biology seems to take advantage of, is to have non-collocated sensors (i.e., in the muscle and tendon instead of the joint) and use fusion of sensory information from actuators and tendons to predict joint angles. It is interesting to note that biological systems do not seem to have dedicated sensors that explicitly and uniquely encode joint angles. Instead, they have sensors for muscle (actuator) lengths and velocities (called muscle spindles, Figure 4.1 orange; Crowe and Matthews, 1964) and for tendon tensions (called Golgi tendon organs, Figure 4.1 blue; Appenteng and Prochazka, 1984) 9 . In previous chapters we established that a functional (yet indirect) relationship exists between sensory states, in general, and kinematic states (like posture; Valero-Cuevas, 2016; Zajac, 1989; Hagen and Valero-Cuevas, 2017). It is therefore speculated that these sensory signals may be inte- grated through their spinal and supraspinal projections to form internal representations of expected or virtual limb position (Scott and Loeb, 1994; Dimitriou and Edin, 2008; Van Soest and Rozen- daal, 2008; Kistemaker et al., 2013). The existence (and possible use) of this indirect relationship between sensory states and kinematic states in biology implies it may be possible to use sensory fusion in tendon-driven robots to infer joint angles from actuator (e.g., motor angles) and structural 9 There are additional biological sensors that detect stretch in the skin and synovial capsule, but these do not directly encode joint position either (Kandel and Schwartz, 2000). 97 α l m ,l m . f T θ Figure 4.1: Example of types of biological sensors present in a muscle. From the spinal cord -motorneurons project to the muscle fascicles to activate it. From there, the muscle spindles are responsible for sensing the resulting muscle fascicle lengths and velocities (l m & _ l m , orange) while the Golgi tendon organs are responsible for detecting tendon tension (f T , blue). These sensory signals may be integrated through their spinal and supraspinal projections to form internal representations of expected or virtual limb position (; Scott and Loeb, 1994; Dimitriou and Edin, 2008; Van Soest and Rozendaal, 2008; Kistemaker et al., 2013). Note that there are additional biological sensors that detect stretch in the skin and synovial capsule not shown here, but these do not directly encode joint position either (Kandel and Schwartz, 2000). (e.g., tendon tensions) sensors, thereby removing the need for on-location joint angle encoders. While it is sometimes possible to derive analytical relationships among tendon tensions, motor rotations, and joint posture given the precise equations for the kinematics and dynamics (see Chap- ter 3), in practice it is often impractical or impossible to obtain accurate and time-invariant models of such nonlinear dynamical systems (Bongard et al., 2006; Marjaninejad et al., 2019b). Further- more, even if an accurate model of the system were available, these relationships (i) would not generalize across changes in mechanical designs or tasks and (ii) will become increasingly inac- curate as the plant suffers mechanical changes due to either damage or normal wear and tear (Palli et al., 2012). Therefore, data-driven systems that can efficiently create mappings between sensory 98 information are preferred in practical applications (Bongard et al., 2006; Marjaninejad et al., 2018; Kwiatkowski and Lipson, 2019). Here we introduce a framework to train artificial neural networks (ANNs) from limited expe- rience via motor babbling to be able to predict joint angles from sets of non-collocated sensory information. As a proof of concept, we simulated an inverted pendulum (controlled by two motors that pull on tendons with nonlinear elasticity), trained ANNs on different sets of sensory infor- mation (i.e., actuator and/or tendon tension data) for different durations of motor babbling and different network architectures, and evaluated the performance of the ANNs and their ability to generalize their performance to different unlearned movement tasks. 4.3 Material and Methods The main goal of this experiment is to train ANNs to predict joint angles from 4 different sets of non-collocated sensory information (~ x i sens ) generated from limited-experience motor babbling to see whether the inclusion of tendon tension information improves the joint angle prediction accu- racy. We will first discuss the dynamics of the proposed plant and the compliance of the tendons that actuate it. Then we will discuss how we generate training data through the use of limited- experience motor babbling (strategic nullspace searching) and how the ANNs were constructed and trained to predict joint angles from the four sets of sensory information. Lastly, we will dis- cuss how four different movements (and the resulting sensory information) were generated to be able to test the generalizability of these ANNs. 4.3.1 Definition of the Plant and its Dynamics In order to determine the utility of observing different sets of sensory information in a tendon- driven system it is important to have a consistent model across trials—such as a standard numerical 99 rm rm rj f T,2 ( θ j ,θ m,2 ) f T,1 (θ j ,θ m,1 ) θ j θ m,2 θ m,1 τ 1 τ 2 Figure 4.2: Schematic of tendon-driven system with 1 kinematic DOF and 2 degrees of actuation (motors) that pull on tendons with nonlinear elasticity (creating a tension, f T;i ). The motors are assumed to be backdriveable with torques ( i ) as inputs. simulation. As we aimed to conduct a thorough and systematic experiment, it was also impractical to use a physical system that is prone to imperfect modelling and time-varying changes to physical parameters. For those reasons, we have simulated a simple 1 degree of freedom (DOF) tendon- driven system with 2 actuators that pull on tendons with nonlinear stiffness (Fig. 4.2) such that we can know/control the parameters that govern the system’s dynamics and we can reliably conduct many experiments on the same plant. We modelled the actuators as brushed DC motors with no gearing to allow for the motors to be backdriveable and we considered the input to be motor torques ( i ). Similar to the approach taken in Palli et al. (2007), the tension on a tendon (f T;i , Eq. 4.1a) was modeled to be an exponential function of tendon deformation (l T;i ) with positive scaling coeffi- cient (k T ) and rate constant (b T ) to fit the shape and slope of this relationship. Tendon deformation is calculated as the difference between joint angle ( j ) and motor angle ( m;i ) excursions. f T;i ( m;i ; j ) = 8 < : k T exp(b T l T;i ) 1 ; (l T;i 0) 0; (l T;i < 0) (4.1a) (4.1b) where l T;i = r m m;1 r j j ; i = 1 r m m;2 +r j j ; i = 2 andb T > 0; k T > 0 are shape constants. 100 8 > > > < > > > : j = 1 I j h D j _ j G( j )+r j f T;1 ( j ; m;1 )f T;2 ( j ; m;2 ) i m;i = 1 I m;i h D m _ m;i r m f T;i ( j ; m;i )+ i i (fori2f1;2g) (4.2a) (4.2b) Therefore, the equations of motion for this system without contact are given by Eq. 4.2 whereG is the torque due to gravity 10 , andI,D, andr represent the moment of inertia, damping coefficient, and moment arm of either the joint or the motors (denoted by the subscriptsj andm, respectively). The values of the parameters used in Eqs. 4.1a–4.2 have been provided in Table 4.1. We can then rewrite the system of equations for contactless dynamics in its state space representation, (Eqs. 4.3–4.5), where ~ x = [ j ; _ j ; m;1 ; _ m;1 ; m;2 ; _ m;2 ] T , ~ u = [ 1 ; 2 ] T , and ~ y = h(~ x) is the desired output (e.g.,y = j ). 8 < : _ ~ x =f(~ x) +g(~ x)~ u ~ y =h(~ x) (4.3a) (4.3b) f(~ x) = 0 B B B B B B B B B B B B B B B B B @ x 2 1 I j h D j x 2 G(x 1 ) +r j f T;1 (x 1 ;x 3 )f T;2 (x 1 ;x 5 ) i x 4 1 I m;1 D m x 4 r m f T;1 (x 1 ;x 3 ) x 6 1 I m;2 D m x 6 r m f T;2 (x 1 ;x 5 ) 1 C C C C C C C C C C C C C C C C C A (4.4a) (4.4b) (4.4c) (4.4d) (4.4e) (4.4f) 10 Torque due to gravity:G( j ) =m j gL CM sin( j ) 101 g(~ x) = 0 B B B B B B B B @ 0 0 0 0 0 0 1=I m;1 0 0 0 0 1=I m;2 1 C C C C C C C C A (4.5) However, this is a model of a physical system that is allowed to come into contact with its boundaries (90 from vertical) which will require a separate set of dynamical equations to be enforced during contact (or impact). To approximate contact dynamics at the boundaries, we in- cluded “hard stops” whenever the joint angle attempted to leave the range of motion when obeying the dynamics that assumed no contact (Eq. 4.2). This is equivalent to preventing any rotation past the boundaries while introducing a restoring joint torque that is equal but opposite to any torque applied against the boundary (s.t. the right hand side of Eq. 4.2a is zero when pushing on the boundaries at=2). This approach allowed us to avoid using high impedance boundary func- Table 4.1: List of plant parameters for simple 1 DOF pendulum joint, actuated by two backdrive- able motors that pull on nonlinearly compliant tendons. Tendon stiffness values (k T andb T ) were conservatively chosen to cause large, but reasonable tendon deformations for some maximum ten- sion (See Section 4.3.2 for further explanation). Additionally, moment arm values (r j andr m ) were approximated to reflect the overall scale of the pendulum. All other parameters were either taken or modified from those provided in Palli et al. (2008). Plant Parameters Pendulum Inertia,I j (kg m 2 ) 1:15 10 2 Center of Mass,L CM (m) 0.085 Pendulum Length,L (m) 0.30 Pendulum Mass,m j (kg) 0.541 Tendon Scaling Coefficient,k T (N) 100 Tendon Rate Constant,b T (m 1 ) 20 Joint Damping,D j (Ns/m) 1 10 3 Joint Moment Arm,r j (m) 0.05 Motor Inertia,I m;i (kg m 2 ) 6:6 10 5 Motor Damping,D m (Ns/m) 4:62 10 3 Motor Moment Arm,r j (m) 0.02 102 tions to model contact dynamics as these would require a variable time step integrator for stability, but it did require a small enough time step to handle the “impact” (t = 10 3 sec). After usingEq. 4.2 to calculate~ x at timet n+1 through Forward Euler integration: j j (t n+1 )j> 2 =) j (t n+1 ) = sgn j (t n+1 ) 2 =) _ j (t n+1 ) = j (t n+1 ) j (t n ) t n+1 t n (4.6a) (4.6b) Note that if the system is already at a boundary (j j (t n )j = =2), any attempt to go into the boundary (i.e.,j j (t n+1 )j>=2) will result in the joint staying at that boundary with zero angular velocity and acceleration, while it is free to rotate back into the middle of the range of motion. And while the dynamics of the joint angle must account for this impact, the motor dynamics only “see” this impact through the resulting tendon tensions. 4.3.2 Tendon Stiffness Parameters The parametersk T andb T in Eq. 4.1a determine the shape of the exponential tendon tension- deformation relationship. Defining instantaneous tendon stiffness to be the derivative of Eq. 4.1a with respect to tendon deformation (l T;i ), we find that k T b T approximates tendon stiffness for small deformations (l T;i 0) andb 2 T will heavily influences tendon stiffness for larger deforma- tions (Eq. 4.7). So how should we choose these parameters and how stiff should our tendons be? As we want to explore what types of sensory information are needed to accurately predict joint angles in a compliant tendon-driven system, it is useful to consider the extreme case where tendons are very compliant to address if tendon tension information is crucial for this estimation. More simply put, if knowing tendon tension does not improve joint angle predictions for highly compliant tendon- 103 driven systems (where there is large decoupling between motor and joint behavior), then we would not expect this information to be useful in less compliant systems where motor behavior becomes more coupled to that of the joint. Therefore we will begin in this chapter by exploring a very com- pliant system, but we will consider the effect of increasing tendon stiffness in Chapter 5 as these results will be influenced by our choice of tendon stiffness parameters. We know from previous work that tendon compliance decouples the relationship between muscle and musculotendon in vivo and suspect that for a mechanical system this decoupling makes it difficult to predict joint angles from motor angle and angular velocity measurements. K T;i = @f T;i @(l T;i ) =k T b T exp(b T l T;i )k T b T (1b T l T;i ) (when l T;i << 1) (4.7) Choosing values for these parameters requires careful consideration of (i) the range of tendon tensions that the motors will produce during movements and (ii) the amount by which the tendons should deform as a result. As it is difficult to determine the required tendon tensions (and therefore tendon deformations) for any redundant tendon-driven system (as illustrated in Chapter 2), an alternative approach is to consider a conservative range of joint stiffness values. For a tendon- driven system with nonlinear tendon elasticity we can define joint stiffness as the partial derivative of the net joint torque produced by the tendons with respect to the joint angle (Eq. 4.8). K j = @ @ j h r j f T;1 ( m;1 ; j )f T;2 ( m;2 ; j ) i =k T b T r 2 j exp b T l T;1 + exp b T l T;2 Assuming l T;i 0 fori2f1; 2g =b T r 2 j f T;1 ( m;1 ; j ) +f T;2 ( m;2 ; j ) + 2k T (4.8a) (4.8b) (4.8c) Note that because of the nature of exponential functions and their derivatives, by assuming that tendons are not allowed to go slack (i.e., l T;i 0 fori2f1; 2g), we can rewrite the expression for joint stiffness as a function of tendon tensions. Additionally, if no tension is applied to the 104 tendons, the minimum joint stiffness is given by K min j = 2k T b T r 2 j . Based on the range of joint stiffness values tested in Palli et al. (2008), if we assume a conservative range of [10,50] Nm/rad and consider the case where the pendulum is in equilibrium in the vertical position, then we know that because of the plant’s symmetry that the tendon tensions and deformations must be equal and we can solve for reasonable values ofk T andb T by choosing a conservative range of tendon deformations for a very compliant tendon; l T;i 2 [0; 0:08] cm (Eq. 4.9). 8 > < > : K min j = 10 Nm rad = 2k T b T r 2 j K max j = 50 Nm rad = 2k T b T r 2 j exp b T 0:08 m =) 8 > > < > > : b T = ln(50=10) 0:08 m 20 m 1 k T = 10 Nm 2b T r 2 j 100 N (4.9) Lastly, it should be noted that for this choice ofk T andb T , tendon tension reaches 400 N at the maximum tendon deformation ( 0:08 m, by Eq. 4.1a). If we assume a cylindrical tendon with a diameter (d) of 0.001 m and a slack length (l T;s;i ) between 0.01–0.04 m, then Young’s modulus (E) at the maximum deformation can range from 0.127 to 0.509 GPa (i.e., rubberband- like material). E = K T;i (0:08 m)l T;s;i CSA = (9906:0 N/m)l T;s;i 2:5 10 7 m 2 2 [0:127; 0:509] GPa (4.10) However, in Chapter 5 we evaluate the performance of these ANN joint predictor algorithms as tendon stiffness increases. Two additional choices for the parameters of k T and b T are derived using the above technique for maximum tendon deformations around 0.0267 m and 0.0125 m (i.e., 33% and 16% of the current conservative maximum deformation, respectively). Assuming similar tendon architecture, the range of Young’s moduli for these stiffness parameters when 400 N are applied to the tendons will increase 2.6x and 5.2x, respectively. These values are more consistent with the observed range of Young’s moduli for physiological tendon (1.2–1.7 GPa), 105 allowing observations to be made about the utility of physiological tendon sensors (i.e., Golgi tendon organs) and their role in sensory fusion (Bennett et al., 1986; Zajac, 1989; Pollock and Shadwick, 2017). 4.3.3 Description of Motor Babbling As previously discussed, in addition to designing an algorithm that can accurately predict joint angles from non-collocated sensory signals, it is important that it do so in a data- and time-efficient manner. Minimizing the amount of time and data that is needed to train these ANNs (i) decreases the chances of significant wear and tear on the physical system and (ii) increases the usefulness of the algorithm by promising quick and accurate state estimation (i.e., no long calibration times needed). In order to efficiently learn a mapping from a particular sensory set (~ x i sens ) to joint angles, we performed a motor babbling trial whereby we (i) passed a series of random input torques to the motors, (ii) recorded all subsequent sensory information, and then (iii) trained an ANN on that specific sensory set to predict joint angle (Fig. 4.3). The motor babbling input torques are con- tinuous signals generated from low frequency, band-limited white noise (1-10 Hz) that uniformly sample the input torque range (0-10 Nm). Additionally, as the motors effectively fight each other in a game of “tug of war” where the joint only moves if there is a net torque, we determined that the most efficient form of motor babbling would be one that simultaneously explores the task space (i.e., joint angle) and its nullspace (i.e., joint stiffness). Therefore, the babbling signal prescribed to one motor was made to be similar to the signal prescribed to the other such that the difference between them would form a normal distribution with zero mean and a standard deviation of 0.5 Nm (or 5% the maximum input level). As a result, this approach would effectively sample the configuration space near the nullspace whenever the input levels were consistent with movement (i.e., the tendon tensions were offset similar to cocontraction in biological systems). 106 τ min τ max 1s Input Generator Plant τ bab θ j,exp x sens i i ANN x sens i θ j,pred i (Generate Motor Babbling) ( Train Artifical Neural Networks ) on Four Main Sensory Sets to Predict Joint Angle Figure 4.3: Proposed setup for training ANNs on motor babbling. Random input torques were generated from low frequency, band-limited white noise (1-10 Hz) chosen such that the difference between the two signals has a normal distribution (0 0:5 Nm). These motor babbling signals are passed through the plant and all subsequent sensory information is recorded. Lastly, an ANN is trained on a particular set of sensory information (~ x i sens ) to predict joint angle ( i j;pred ). To generate motor babbling signals of a particular duration, the signals are first divided into 50 ms windows. For each 50 ms window, a random motor torque is uniformly selected from the range of inputs and assigned to the first motor (Figure 4.4 A). Values for the second motor for each window are then randomly selected from normal distributions centered around the first motor’s input values with a standard deviation of 0.5 Nm (Figure 4.4 B). These discontinuous, piecewise- constant input signals are then filtered with a 50 ms moving average, finite-impulse response filter both forward and backwards to produce random motor babbling signals that are smooth, low fre- quency ( 10 Hz), and slightly correlated with each other (Figure 4.4 C,D). These motor babbling inputs are then used to drive the plant in a feedforward simulation where the resulting joint angles 107 300 0 100 200 10 0 τ min τ max (A) Find Uniform Random Inputs Motor 1 (Every 50 ms) (B) Find Normal Random Inputs for Motor 2 Near Motor 1 (D) Apply 50 ms Moving Average Filter (Reverse) (C) Apply 50 ms Moving Average Filter (Forward) Motor Input Torques (Nm) Time (ms) Figure 4.4: Example of how 300 ms of motor babbling signals are generated. (A) Random motor torque inputs are uniformly sampled from the range of possible inputs and assigned to 50 ms windows for motor 1 (red). (B) Then values for motor 2 (blue) are selected for each window from a normal distribution centered around the values for motor 1 with a standard deviation of 0.5 Nm (5% of the range of maximum input level). These discontinuous, piecewise signals are then filtered using a forward (C) and backward (D) finite impulse response moving average filter with a filter lengths of 50 ms. This results in correlated band-limited, low frequency ( 10 Hz) white noise motor babbling signals. are recorded along with all of the motor states (angles, angular velocities, and angular accelera- tions) and the tendon tension states (including their first and second derivatives). 4.3.4 Training & Testing Artificial Neural Networks For each babbling trial, four different sets of sensory information were generated from the collected motor and tendon tension states and used to train ANNs to predict joint angles. As a baseline set, we first consider the set of all motor and tendon tension states as in Eq. 4.11 (called the set of All Available States). 108 All Available States ~ x 1 sens = h ~ T m _ ~ T m ~ T m ~ f T T _ ~ f T T ~ f T T i T 2R 12 (4.11) We then consider the Bio-Inspired Set, which observes motor position and velocities and tendon tensions (Eq. 4.12). This set is reminiscent of the sensory signals available to biological systems from the non-collocated muscle spindles and Golgi tendon organs, respectively. Bio-Inspired Set ~ x 2 sens = h ~ T m _ ~ T m ~ f T T i T 2R 6 (4.12) The last two sets, given by Eqs. 4.13 & 4.14, consider the sensory sets that do not include tendon tensions in any form. The first considers motor position and velocities only (this set parallels a hypothetical biological system that only uses spindle information to determine joint posture; called the set of Motor Position and Velocity Only), and the second incorporates motor acceleration to see if it can provide useful information about the plant dynamics not captured by kinematics alone (called the set of All Motor States). Motor Position and Velocity Only ~ x 3 sens = h ~ T m _ ~ T m i T 2R 4 (4.13) All Motor States ~ x 4 sens = h ~ T m _ ~ T m ~ T m i T 2R 6 (4.14) For each of these sensory sets, an ANN was trained to predict joint angles. To do this, MAT- 109 LAB’s Deep Learning toolbox was used to reliably create feedforward neural networks while main- taining direct control over the parameters of the network 11 . Specifically, we were able to control (i) the number of nodes in our hidden layer, (ii) the way in which the weights and biases of the activa- tion functions were initialized, and (iii) what optimizer was use. This type of neural network uses backpropagation to approximate the gradient of the cost function (the mean squared error) with re- spect to the weights after each input-output pass in order to tune the weights of the networks until the cost is (locally) minimized. For each ANN created, the number of input layer nodes equals the number of states in the sensory set and the number of output layer nodes is one. Each network will have one hidden layer and the number of hidden layer nodes will be explored in later sections to determine the effect that it has on the performance of each network. The function used to initialize the weights and biases of the network was changed from the de- fault function (Nguyen-Widrow initialization algorithm) to one that ensure the weights and biases are initialized randomly each trial. The Nguyen-Widrow algorithm is designed to reduce training time by distributing the initial weights and biases in such a way that the active region of each neu- ron in the hidden layer is evenly distributed across the input space. As the goal of this experiment is to compare how well and how quickly ANNs trained on different sensory sets can learn to pre- dict joint angles, the Nguyen-Widrow algorithm is a poor candidate for initialization function as it may influence the average performance differently across ANNs trained on the four sensory sets. The optimizer was chosen to be the Levenberg-Marquardt algorithm (a nonlinear least squares op- timization technique) as it is very robust for minimizing mean squared error (MSE). As we wish to compare how well these ANNs trained on the four sensory sets are able to predict the joint angle for a simple 1 DOF system, we will use the mean absolute error (MAE) to discuss performance, but for systems with higher DOFs a mean squared error (or root mean squared error, RMSE) would be more appropriate. Therefore, this choice of optimizer is appropriate to find the (locally) optimal performance for these ANNs. 11 https://www.mathworks.com/help/deeplearning/ref/feedforwardnet.html 110 For each babbling trial and each sensory set, the babbling data is randomly divided into three sets; training (70%), validation (15%), and testing (15%). The training sets will be used to cal- ibrate the weights of the ANN using backpropagation, while the validation data will be used to ensure that the network does not overfit to the data. Validation checks are performed after each epoch (i.e., one full pass of the training data) and will terminate the training if (i) the performance has gotten worse for 6 consecutive epochs or (ii) if the gradient of the performance is below a cer- tain threshold (10 7 , i.e., if the performance curve has flattened out and is near a minimum). The number of epochs that the networks can train on can also be modified to prevent overfitting (as well as to reduce training time), but because we are interested in the best performance possible by ANNs trained on these sensory groups, the epoch limit was set to 10,000 to allow (most) networks to con- verge by means of the aforementioned validation checks instead. Once a network has converged, it then uses the testing data to calculate the performance of each network (effectively testing on data that has been previously “unseen” by the ANN). Because this training algorithm utilizes time histories of sensory information, it is to be expected that each ANN perform as well on the testing data set as it does on the training data set, because there will be little difference between randomly chosen data points that may be separated by as little as 1 ms in time (i.e., if a ANN was trained on data collected at timet then testing on data collected at timet + 0:001 s should not be all that different in a continuous system and the performance should be similar to the testing performance). Therefore it will be critical that we additionally test how each network performs when generalizing to different movement tasks. The next section will discuss how these generalization movements were generated. 4.3.5 Defining Different Movements to Test Generalizability While it is important to understand how these limited experience ANNs performed on the test data (random 15% sample of babbling data) to identify whether different sensory sets can be used to predict joint angles sufficiently, it is perhaps more important to understand if these networks can 111 generalize to different types of movements. To adequately address each ANNs ability to gener- alize, we needed to identify movement tasks that were representative of most typical movements. An enabling feature of tendon-driven system with tendons with nonlinear stiffness is the ability to control joint angle independently of joint stiffness (K j , Eq. 4.8) by choosing different tendon tensions in the nullspace of the joint dynamics. Therefore, by defining four different movement tasks where joint angle and stiffness are prescribed either sinusoidal or random point-to-point tra- jectories within the range of these values, we are able to generate movement tasks that categorized most typical movements. In the next section we will discuss how a feedback linearization algorithm can be used to con- trol both the joint angle and stiffness of this system in order to generate the states associated with each of these generalization movements, but for now it is important to know that because the con- trol policy will depend on up to the fourth derivative of joint angle and the second derivative of joint stiffness, these reference trajectories must be at least differentiable up to the fourth derivative to avoid unwanted transients in the control (i.e., beC 4 differentiable). This will not be an issue for sinusoidal trajectories, but when prescribing point-to-point tasks, this will become more impor- tant. To address this, a modified minimum-jerk trajectory was derived such that it was possible to smoothly leave from and arrive at a point in either joint angle or stiffness space with zero velocity, acceleration, jerk, and snap (i.e., all derivatives up to the fourth order will be zero, Eq. 4.15–4.19; Flash and Hogan, 1985). For a desired output variabley to transition between some initial (i) and final (f) point: y =y ;i + (y ;f y ;i ) 126 5 420 6 + 540 7 315 8 + 70 9 (4.15) dy d = (y ;f y ;i ) 630 4 2520 5 + 3780 6 2520 7 + 630 8 (4.16) d 2 y d 2 = (y ;f y ;i ) 2520 3 12600 4 + 22680 5 17640 6 + 5040 7 (4.17) d 3 y d 3 = (y ;f y ;i ) 7560 2 50400 3 + 113400 4 105840 5 + 35280 6 (4.18) 112 d 4 y d 4 = (y ;f y ;i ) 15120 151200 2 + 453600 3 529200 4 + 211680 5 (4.19) where = (tt i )=(t f t i ) and2f j ;K j g. By limiting the amount of time it takes to transition between points (t f t i ) to a conservative (2 2 Hz) 1 = 0:25 seconds we can limit the frequency content of the reference trajectory to be below 2 Hz. The output variable can then be held constant for a given duration (minus the transition time), before transitioning to another point. Therefore, we define the four movement trajectories used to test generalization trajectories as follows. When both joint angle and joint stiffness were made to follow sinusoidal trajectories, the joint angle was prescribed a sinusoidal trajectory of45 from vertical with a frequency of 1 Hz, while the joint stiffness was prescribed a cosine trajectory with twice the frequency such that max- imum stiffness occurred at the extremes of the movement (or minimum stiffness when swinging across the configuration space, Eq. 4.20, Figure 4.5, left). For this and all other trajectories, the range of joint stiffness values was chosen to be [20,50] Nm/rad. The length of this reference trajec- tory was chosen to be 10 seconds (10 periods of joint rotations), but once the controller converged on this trajectory, so too did the states associated with it. Therefore, because the sensory states will be periodic, the performance of each ANNs will be periodic as well and only one period is needed to capture the performance behavior. However, we will use the last three periods to ensure we capture the average behavior. Angle Sinusoidal / Stiffness Sinusoid r j = 4 sin(2t) K r j = 50 + 20 2 50 20 2 cos(4t) (4.20a) (4.20b) When the joint angle was made to follow a sinusoidal trajectory but the joint stiffness was made to follow a point-to-point task, the joint angle trajectory was prescribed as described above 113 (1 Hz oscillation,45 from vertical), and the joint stiffness point-to-point task was chosen such that (i) the point-to-point values uniformly sampled the stiffness range and (ii) the step duration was equal to 3 times the period of the joint angle trajectory (Figure 4.5, middle left). This ensured that the maximum positive angular velocity has two periods where it does not coincide with the transition to another stiffness point. Therefore, for each new random joint stiffness value, Eq. 4.15 was used to generate a smooth transition from the previous value with the appropriate boundary conditions, where it will be held constant for 2.75 seconds before transitioning to the next value (step duration minus the transition time, Eq. 4.21). To adequately sample the joint stiffness space, we will complete 100 point-to-point steps for this trajectory (300 seconds total). Angle Sinusoidal / Stiffness Point-to-Point r j = 4 sin(2t) K r j = 8 > > > < > > > : Eq. 4.15 for smooth transitions Otherwise, constant hold phase at random stiffness values for 3 periods minus transition time (2.75 seconds) (4.21a) (4.21b) Similarly, when the joint stiffness was made to follow a sinusoidal reference trajectory, but the joint angle was made to perform point-to-point tasks, we prescribed a 1 Hz cosine trajectory for the joint stiffness that spanned the joint stiffness range and chose random point-to-point joint angle values that uniformly sampled the entire range of motion with a step duration 3 times the period of the joint stiffness trajectory (Eq. 4.22, Figure 4.5, middle right). As before, to adequately sample the joint angle space, we will complete 100 point-to-point tasks for this trajectory (300 seconds). 114 Angle Point-to-Point / Stiffness Sinusoidal r j = 8 > > > < > > > : Eq. 4.15 for smooth transitions Otherwise, constant hold phase at random angle values for 3 periods minus transition time (2.75 seconds) K r j = 50 + 20 2 50 20 2 cos(2t) (4.22a) (4.22b) Lastly, when both joint angle and stiffness were made to follow point-to-point trajectories, the random values were uniformly selected from the full range of motion and the joint stiffness range, respectively (Figure 4.5, right). The step duration was set to 1 second (minus the transition time), to ensure that the plant had ample time to converge to the point in joint angle and joint stiffness space. As described above when the joint angle and stiffness followed sinusoidal trajectories, once the behavior of the plant converges, so too will the performance. Therefore this step duration is adequate to capture the behavior of a “ramp-and-hold” task because the controller was able to converge to this static point within the alotted step duration. For this trajectory, 200 separate point-to-point tasks were completed (200 seconds total). Angle Point-to-Point / Stiffness Point-to-Point r j = 8 > < > : Eq. 4.15 for smooth transitions Otherwise, constant hold phase at random angle values for 1 second minus transition time (0.75 seconds) K r j = 8 > < > : Eq. 4.15 for smooth transitions Otherwise, constant hold phase at random stiffness values for 1 second minus transition time (0.75 seconds) (4.23a) (4.23b) 115 Point-to-Point Point-to-Point Point-to-Point Sinusoidal Point-to-Point Sinusoidal Sinusoidal Sinusoidal Joint Angle (Deg) Joint Stiffness (Nm/rad) 0 90 -90 50 20 1s Figure 4.5: Example plots of four different types of reference trajectories where joint angle and joint stiffness are either varied sinusoidally or in a point-to-point task. For point-to-point tasks, transitions are limited to 0.25 seconds (2 Hz cutoff) and are designed in a way that they are con- tinuously differentiable up to the fourth derivative (i.e., they leave and arrive each point with zero velocities, accelerations, jerks, and snaps). 4.3.6 Controlling for Joint Angle and Stiffness via Feedback Linearization In order to test how well each ANN generalizes to the four movement tasks derived in the pre- vious section, we need the sensory sets associated with each movement. To do this, we will control the system so that it adequately follows the reference trajectories described above and record the resulting sensory information. As we are controlling both the position and stiffness of the joint, control is possible via a feedback linearization algorithm (Palli et al., 2007). If we consider the outputs of the system to bey = h(~ x) = [ j ;K j ] T , then feedback linearization completely trans- forms the nonlinear system of equations, (Eq. 4.3), into a linear system that can be controlled by feedback. A condition for the use of this tool is that the sum of the elements of the relative vector degree be equal to the number of states so that there is no portion of the dynamics that is uncontrollable 116 by feedback (i.e., there are no zero dynamics). The relative degree of one of the output variables is found by differentiating that variable until the input variable appears. We find that we must differ- entiate the joint angle 4 times and the joint stiffness 2 times in order for the input torques to appear, which corresponds to a total relative degree of 6 for a system of equations with 6 states (Eqs. 4.24– 4.26). Note thatL n f h j andL n f h K j represent then-th Lie derivatives of the functionsh j (~ x) = j andh K j (~ x) =K j alongf(~ x), respectively. Additionally,L g L 3 f h j (~ x) andL g L f h K j (~ x) are the Lie derivatives ofL 3 f h j (~ x) andL f h K j (~ x) alongg(~ x), respectively, whereL g h j (~ x),L g L i f h j (~ x) for i2f1; 2g, andL g h K j (~ x) equal zero. y j =h j (~ x) = j _ y j = @h j @~ x T f(~ x) +g(~ x)~ u =L f h j (~ x) + : 0 L g h j (~ x)~ u = _ j y j = @L f h j @~ x T f(~ x) +g(~ x)~ u =L 2 f h j (~ x) + : 0 L g L f h j (~ x)~ u =f 2 (~ x) y [3] j = @L 2 f h j @~ x T f(~ x) +g(~ x)~ u =L 3 f h j (~ x) + : 0 L g L 2 f h j (~ x)~ u = @f 2 @~ x T f(~ x) y [4] j = @L 3 f h j @~ x T f(~ x) +g(~ x)~ u =L 4 f h j (~ x) +L g L 3 f h j (~ x)~ u = f(~ x) T @ 2 f 2 @~ x 2 + @f 2 @~ x T @f @~ x ! f(~ x) +g(~ x)~ u (4.24a) (4.24b) (4.24c) (4.24d) (4.24e) (4.24f) y K j =h K j (~ x) =K j _ y K j = @h K j @~ x T f(~ x) +g(~ x)~ u =L f h K j (~ x) + : 0 L g h K j (~ x)~ u = @K j @ j _ j + @K j @ m;1 _ m;1 + @K j @ m;2 _ m;2 y K j = @L f h K j @~ x T f(~ x) +g(~ x)~ u =L 2 f h K j (~ x) +L g L f h K j (~ x)~ u = f(~ x) T @ 2 K j @~ x 2 + @K j @~ x T @f @~ x ! f(~ x) +g(~ x)~ u (4.25a) (4.25b) (4.25c) (4.25d) (4.25e) 117 Table 4.2: List of variables for Eq. 4.28 that solve an algebraic Ricatti equation for continuous time. Derived in Palli et al. (2008) and reproduced here. Coefficients forEquation4.28 Output Variable (y) c y;0 c y;1 c y;2 c y;3 Joint Angle ( j ) 3162.3 1101.9 192.0 19.6 Joint Stiffness (K j ) 316.2 25.1 We then rewrite the system using the derivatives of the output where the input first appears (Eq. 4.26). By choosing ~ u to satisfy Eq. 4.27, we can choose ~ v = [v j ;v K j ] T such that the now linear system stabilizes around the prescribed reference trajectories (Eq. 4.28). The values for this equation were derived solving an algebraic Ricatti equation for continuous time (Palli et al., 2008) and have been reproduce below (Table 4.2). Sample plots for the results of following the generalization trajectories with this feedback linearization controller are provided in Figure 4.6 where the joint angle output is plotted alongside the positions and velocities of the motors, tendon tensions, and motor input torques. 8 < : y [4] j =L 4 f h j (~ x) +L g L 3 f h j (~ x)~ u y K j =L 2 f h K j (~ x) +L g L f h K j (~ x)~ u (4.26a) (4.26b) ~ u = " L g L 3 f h j (~ x)~ u L g L f h K j (~ x)~ u # 1 v j v K j " L 4 f h j (~ x) L 2 f h K j (~ x) #! (4.27) 8 < : v j = r[4] j +c j ;3 ( r[3] j y [3] j ) +c j ;2 ( r j y j ) +c j ;1 ( _ r j _ y j ) +c j ;0 ( r j y j ) v K j = K r j +c K j ;1 ( _ K r j _ y K j ) +c K j ;0 (K r j y K j ) (4.28a) (4.28b) 118 Sinusoidal Sinusoidal Joint Stiffness: Joint Angle: 100 200 300 400 Tendon Tensions (N) 2 4 6 8 Motor Inputs (Nm) Joint Angle (Deg) Motor 1 90 180 0 270 Pendulum Motor 2 90 180 270 360 Joint Angle (Deg) Joint Angle (Deg) 0 90 -90 Point-to-Point Sinusoidal Point-to-Point Sinusoidal −1500 0 1500 Angular Velocity (Deg/s) −1500 0 1500 Angular Velocity (Deg/s) Point-to-Point Point-to-Point −400 0 400 Angular Velocity (Deg/s) 1s 119 Figure 4.6: Sample plots comparing the performance of the feedback linearization algorith when following the four reference trajectories of interest (where joint angle and joint stiffness are varied sinusoidal or with a point-to-point task). The resulting motor positions and velocities, tendon tensions, and motor input torques are provided beneath the joint angle output of each trajectory. It should be noted that the motor kinematics are not always coupled to the behavior of the pendulum (especially when the joint angle is constant during a point-to-point task but the joint stiffness is varied sinusoidally, middle right), suggesting it would be difficult to imply joint angle from motor measurements alone. It is important to note that the use of a feedback linearization controller (that relies on an accurate model of the system) was used to (i) prescribe desired movement trajectories and (ii) fully observe the internal sensory states. As such, it is not a part of the proposed framework to build ANNs to predict joint angle but instead a tool used to generate systematically varied movements and the associated sensory sets needed to test the performance and generalizablity of the proposed ANNs. 4.3.7 Description of Experiments With the general framework for training an ANN on a given sensory set defined, we must now consider how (i) motor babbling duration and (ii) number of hidden layer nodes affect the performance of these ANNs. The general setup for either experiment is shown in Figure 4.7 where the four sensory sets (Eqs. 4.11–4.14) generated from the four generalization movements (Eqs. 4.20–4.23) are used to test the performance of ANNs trained on motor babbling information where either the duration of the babbling or the number of hidden layer nodes are varied. To observe the effect (if any) that the duration of the motor babbling has on the performance of ANNs trained on these four sensory groups, we will select babbling durations between 1–25 seconds (f1g[f2:5; 5;:::; 25g) and conduct 25 motor babbling experiments for each duration chosen. For each babbling duration, the sensory data collected from each of the 25 babbling trials will be divided into the four sensory sets and used to train new ANNs. For each of the four sensory 120 Repeat for N Trials ( ) ( ) K r θ r Desired Movement Task θ j K j Sin P2P Sin P2P Plant Feedback Linearization Controller + - Prediction Errors θ j,exp 1 θ j,pred 4 θ j,pred x sens x sens τFBL . . . . . . 1 4 Figure 4.7: Proposed experimental setup for a single choice of either babbling duration (Exper- iment 1, N = 25) or the number of hidden layer nodes (Experiment 2, N = 10). For either experiment, for each choice of the independent parameter,N motor babbling experiments are con- ducted and N ANNs are trained (See. Figure 4.3). The performance of each of these networks will be determined by their ability to generalize to different movements (where joint angle and/or stiffness are prescribed either sinusoidal (Sin) or point-to-point (P2P) trajectories; See Figure 4.5). A feedback linearization controller then calculates the input torques needed to produce the desired movements (See Section 4.3.6), which are then passed through the plant to produce the experimen- tal joint angle ( j;exp ) as well as the four sensory sets of interest (~ x i sens ). These sets are then passed through their corresponding ANNs that were trained on babbling data to predict joint angle ( i j;pred ). The prediction errors for each network are then averaged over all trials, and the performance as a function of the independent parameter can then be evaluated. sets, the performance of each ANN for each of the four generalization movements is calculated as the mean absolute error between the actual and predicted joint angles and represents how well the ANN generalized to each task. For each babbling duration, the average mean absolute error for each sensory set is calculated for each of the four generalization movements. For this experiment, we initially consider 15 hidden layer nodes as previous work has shown that to be sufficient when controlling the joint angles of a tendon-driven system with rigid tendons (Marjaninejad et al., 2019a). The results from this experiment (discussed in more detail below) lead to the observations that (i) average performance converged after 7.50–10 seconds when using 15 hidden layer nodes and (ii) that for each sensory set, the standard deviation generally decreased as babbling duration increased. From these observations, a separate experiment was conducted where the babbling duration was fixed at 15 seconds while the number of hidden layer nodes was swept from 1–19 with a resolution of 2 nodes to observe how this performance may change if the network architecture 121 changed as well. We chose to use 15 seconds of motor babbling instead of observed 7.5–10 second threshold to provide a safety factor so that changes in performance could only be attributed to changes in the number of hidden layer nodes and not because the choice of babbling duration was near the threshold for “good enough” performance. Because the standard deviations between trials is quite low for any choice of hidden layer nodes, only 10 motor babbling trials were conducted and 10 ANNs were subsequently trained for each sensory set. Again, the performance of each network was determined by how well it generalized to the four movement tasks as described above and the average performance of all 10 trials was calculated for each sensory set and for each choice of the number of hidden layer nodes. 4.4 Results 4.4.1 Sweeping Motor Babbling Duration For the first experiment, where the duration of motor babbling was varied between 1–25 sec- onds (for a fixed number of hidden layer nodes; 15), we find that for all motor babbling durations and for all movement types ANNs trained on tendon tension data (i.e., the set of All Available States and the Bio-Inspired Set), drastically outperform those ANNs that were trained without it (i.e., the sets of Motor Position and Velocity Only and All Motor States). Figure 4.8, which plots the average ANN performance (mean absolute error) of each sensory set against the babbling durations used to train the ANNs for each of the four generalization movements, demonstrates this clearly as the performances of the ANNs trained on tendon tension information are too low to even be seen on the same scale as the performances of the ANNs trained only with motor information. Plotting this same graph on a log scale (Figure 4.9) reveals that the performance of the ANNs trained on tendon tension in addition to motor information perform nearly 2 to 3 orders of magnitude better than those ANNs trained only with motor information and does so (i) for any babbling duration and (ii) across all movements. Interestingly, ANNs trained on the Bio-Inspired Set performed as 122 well as (if not better than) those trained on the set of All Available States, which speaks to both the importance of knowing tendon tension when predicting joint angles from non-collocated sen- sors and the potential lack of utility (or possible redundancy) of higher derivative tendon tension states. Additionally, when comparing the performances of the ANNs trained on motor information only, the inclusion of motor acceleration does initially compensate for the lack of tendon tension information and increases the performance, as expected, when babbling durations are low. But this improvement is not significant and becomes smaller as the duration of motor babbling increases (i.e., motor acceleration information is useful if and only if data is limited, but not nearly as useful as tendon tension information). Obviously, these result are subject to changes in the number of hidden layer nodes. In an effort to elucidate the effect that this floating parameter has on performance, the next experiment will train ANNs with different numbers of hidden layer nodes on the same amount of babbling data. Therefore, a reasonable choice for the duration of motor babbling to be used in the next experiment must be made. We can see from Figure 4.9 that performance converges for some sensory sets with as little as 7.5–10 seconds of motor babbling. However, a closer look at Figure 4.8 reveals that the performances of ANNs trained only on motor information will actually converge after 15 seconds of motor babbling. Therefore, we conclude that for the choice of 15 hidden layer neurons, 15 seconds of motor babbling is sufficient to produce relatively good performance (i.e., near optimal) for ANNs trained on each of the sets of sensory information. To further validate this choice, when exploring the standard deviations of the performances of each sensory set across babbling durations, we see that the standard deviation generally decreases as the babbling duration increases until it saturates after 10–15 seconds of motor babbling (Fig- ure 4.10). It should be noted that because these relationships are plotted on a logarithmic scale, variations in either the average performance (Figure 4.9) or the performance standard deviation (Figure 4.10) for the ANNs trained with tendon tension information will be exaggerated (i.e., these plots show variations relative to each other, when in reality the difference between values in on the 123 5 10 15 20 25 0 5 10 15 20 25 0 0 1 2 3 4 5 Average MAE (deg) Babbling Duration (sec) 0 1 2 3 4 5 (Point-to-Point) (Sinusoidal) Desired Joint Stiffness Trajectory Type (Sinusoidal) (Point-to-Point) Desired Joint Angle Trajectory Type All Available States Bio-Inspired Set Motor Position and Velocity Only All Motor States Figure 4.8: Plots of the average performance (mean absolute error) versus babbling duration (sec- onds) assuming 15 hidden layer nodes. For each babbling duration, 25 ANNs were trained from babbling data and the average error for each generalization trajectory was computed. The ANNs with tendon tension drastically outperform those trained only with motor information. A log scale is provided in Figure 4.9 to discuss the performance of the Bio-Inspired Set relative to the set of All Available States. 124 5 10 15 20 25 0 5 10 15 20 25 0 10 -4 10 -3 10 -2 10 -1 10 0 10 1 Average MAE (deg) Babbling Duration (sec) 10 -4 10 -3 10 -2 10 -1 10 0 10 1 (Point-to-Point) (Sinusoidal) Desired Joint Stiffness Trajectory Type (Sinusoidal) (Point-to-Point) Desired Joint Angle Trajectory Type All Available States Bio-Inspired Set Motor Position and Velocity Only All Motor States Figure 4.9: Plots of the average performance (mean absolute error) versus babbling duration (sec- onds) in log scale assuming 15 hidden layer nodes. For each babbling duration, 25 ANNs were trained from babbling data and the average error for each generalization trajectory was computed. The ANNs trained with tendon tension (dark blue and orange) drastically outperform those trained only with motor information (light blue and purple, with a 3 orders of magnitude improvement when training on 7.5 seconds or longer). We also see that the Bio-Inspired Set will on average per- form as well as our baseline set (All Available States), suggesting that tendon tension in addition to motor positions and velocities are sufficient to predict joint angles. 125 5 10 15 20 25 0 5 10 15 20 25 0 10 -4 10 -3 10 -2 10 -1 10 0 10 1 Average MAE (deg) Babbling Duration (sec) 10 -4 10 -3 10 -2 10 -1 10 0 10 1 (Point-to-Point) (Sinusoidal) Desired Joint Stiffness Trajectory Type (Sinusoidal) (Point-to-Point) Desired Joint Angle Trajectory Type All Available States Bio-Inspired Set Motor Position and Velocity Only All Motor States Figure 4.10: Plots of the standard deviation in the performance (mean absolute error) for each choice babbling duration (seconds) in log scale assuming 15 hidden layer nodes. For each babbling duration, 25 ANNs were trained from babbling data and the average error for each generalization trajectory was computed. We see trends similar to those seen in Figure 4.9, with standard deviation generally decreasing as the duration of babbling increases. Note that because this plotted on a log scale, the peaks for both the All Available States and Bio-Inspired Set at 20 seconds of babbling are on the order of 10 3 to 10 2 , and therefore does not reflect large variations from the average values but more likely noise. 126 order of 10 3 to 10 2 degrees). Therefore, based on the observation that average performance as well as the performance standard deviation for all sensory sets converges after 15 seconds of motor babbling (across all movement tasks), we conclude that 15 seconds is a reasonable value to use to explore the effect that changes in the number of hidden layer nodes may have on performance. 4.4.2 Sweeping Number of Hidden Layer Nodes By fixing the duration of motor babbling to 15 seconds and sweeping the number of hidden layer nodes, it is possible to see if changes in the structure of the ANN will change the performance of these ANNs. Figure 4.11 reveals an interesting consequence of ANN structure and the type of information encoded by these sensory states—networks trained with motor information only actu- ally perform their best with only one hidden layer node and have increasing poorer performance as the number of hidden nodes increases. If we consider the results from the previous experiment (where motor acceleration is only useful when babbling durations are less than 10 seconds), these results present an interesting interpretation of the information encoded by the positions and veloc- ities of the motors; mainly that the best approximation that ANNs trained without tendon tension can provide is captured by a single equation of motor positions and velocities. Adding nodes without providing any additional information will result in worse performance as more floating parameters (i.e., weights and biases) must be set to recapture what was best described by a simple equation, but must do so with limited data. This single equation approximation is reminiscent of the “stiff tendon” approximation used in Chapter 1 (and refuted in Chapter 3), where changes in the kinematically-derived musculotendon excursions are used to approximate muscle fascicle length changes (only here it is the other way around). Conversely, the performance of ANNs trained with tendon tension information increases across all movements when the number of nodes increases. Intuitively, if the number of nodes is too small, features of the data will be discarded as there are fewer available equations to capture it. It 127 5 9 13 11 7 3 17 15 19 1 5 9 13 11 7 3 17 15 19 1 0 0.4 0.8 1.2 Average MAE (deg) No. Hidden Layer Nodes 0 0.4 0.8 1.2 (Point-to-Point) (Sinusoidal) Desired Joint Stiffness Trajectory Type (Sinusoidal) (Point-to-Point) Desired Joint Angle Trajectory Type All Available States Bio-Inspired Set Motor Position and Velocity Only All Motor States Figure 4.11: Plots of the average performance (mean absolute error) versus number of hidden layer neuron assuming 15 seconds of motor babbling. For each choice in the number of hidden layer nodes, 10 ANNs were trained from babbling data and the average error for each generalization trajectory was computed. For any choice in the number of hidden layer nodes, the ANNs trained with tendon tension outperform those trained without it. However, with fewer hidden nodes, the performance of ANNs trained without tendon tension improves while the performance of ANNs trained with tendon tension degrades but still perform best. As expected the performance of the ANNs trained with motor information only worsens as the structure of the ANN becomes more complex because there is not enough information to tune the additional weights. 128 5 9 13 11 7 3 17 15 19 1 10 -4 10 -3 10 -2 10 -1 10 0 10 1 5 9 13 11 7 3 17 15 19 1 Average MAE (deg) No. Hidden Layer Nodes 10 -4 10 -3 10 -2 10 -1 10 0 10 1 All Available States Bio-Inspired Set Motor Position and Velocity Only All Motor States (Point-to-Point) (Sinusoidal) Desired Joint Stiffness Trajectory Type (Sinusoidal) (Point-to-Point) Desired Joint Angle Trajectory Type Figure 4.12: Plots of the average performance (mean absolute error) versus number of hidden layer neuron in log scale assuming 15 seconds of motor babbling. For each choice for the number of hidden layer nodes, 10 ANNs were trained from babbling data and the average error for each generalization trajectory was computed. For any choice in the number of hidden layer nodes, the ANNs with tendon tension outperform those trained only with motor information. However, for fewer and fewer hidden layer nodes, the performance of the ANN trained without tendon tension improves while the performance of ANNs trained with tendon tension data degrade but still per- form best. It can be seen that the performance of the ANNs trained on tendon tension data begin to plateau for 9+ hidden layer nodes. 129 is important, however, to note that while the performance of ANNs trained with tendon tension data decrease with fewer nodes, the performance never becomes worse than that of the ANNs trained without tendon tension. This is further illustrated in Figure 4.12 where the same relationships are plotted on a log scale. Interestingly, it appears that the ANNs trained on tendon tension information will start to lose performance for a feedforward network with less than 9 hidden layer nodes (even though using as little as 3 nodes still results in approximation errors less than 10 2 degrees on average). From this plot we can see that the original choice of 15 nodes for this chapter’s first experiment was reasonable (albeit superfluous) as it provided a safety factor so that changes in motor babbling duration would not have changes in performance that could be attributed to the number of nodes in the hidden layer. 4.4.3 Neural Network Performance Across Sensory Sets and Movements From the results of experiments 1 and 2, it was determined that the performance of ANNs with 15 hidden layer nodes trained on 15 seconds of motor babbling produced reliably good estimates for joint angle from ANNs trained on tendon tension data. While we saw in experiment 2 that for ANNs trained only with motor information (using 15 seconds of motor babbling) the performance is actually best when the number of nodes is lowest (i.e., 1), improvement was (i) not enough to outperform the ANNs trained with tendon tension data and (ii) only slightly better than the performance of ANNs with 10+ hidden layer nodes ( 0:4 vs. 0:8 , respectively). Therefore, in the final chapter, where we consider how changes to movement dynamics or to the motor or tendon properties affect the utility using tendon tension to predict joint angles, we will consider ANN with 15 hidden layer nodes, trained on 15 seconds of motor babbling for all sensory sets. As such, we will discuss in this section the training and performance of ANNs trained for this duration and with this architecture. As previously discussed, for each sensory set, 25 ANNs were trained on the appropriate set 130 Training Performance (RMSE in deg.) vs. Epoch Number 0 2k 4k 6k 8k 10k 10 -4 10 -3 10 -2 10 -1 10 0 10 1 10 2 0 2 4 6 8 10 10 -1 10 0 10 1 10 2 (Average) 0 20 40 60 80 100 10 -3 10 -2 10 -1 10 0 10 1 10 2 All Available States Bio-Inspired Set Motor Position and Velocity Only All Motor States Figure 4.13: Performance (root mean squared error; RMSE in degrees) versus the number of epochs needed to train each ANN. For each of the four sensory sets, 25 motor babbling simulation of 15 seconds were performed to train ANNs with 15 hidden layer nodes. Although it took the ANNs more than 1000 epochs for the performances to converge (even requiring up to 10,000 epochs for the ANNs trained on tendon tension data), the majority of the performance improvement came within the first 20–50 epochs (middle). In fact, the ANNs trained only on motor data (Motor Position and Velocity Only and All Motor States) converge with as little as 6 epochs (as seen by the average plot on the right). Lastly, it appears that learning from motor information may allow for faster learning, but the performance is soon beaten by the ANNs trained with tendon tension (which took longer to learn). of motor babbling data to predict joint angles. Each ANN would refine its weights and biases to increase the accuracy of this prediction by using backpropagation to minimize the mean squared error. The training stopped whenever (i) the performance improvement was sufficiently small between epochs, (ii) the performance actually got worse for 6 consecutive epochs, or (iii) the epoch limit was reached. As the epoch limit was chosen to be sufficiently large, very few ANNs reached this limit before terminating from one of the other two conditions. The left panel in Figure 4.13 shows the training performance after each epoch for all 25 trial for each sensory set. Networks trained on motor information alone terminated their training after less than 1000 epochs on average, while the Bio-Inspired Set and the set of All Available States utilized 2917 and 5284 epochs on average, respectively (Figure 4.14). However, it can be seen from the middle panel in Figure 4.13 that the performance of the ANNs trained with tendon tension already perform 2 orders 131 (15. seconds of Motor Babbling) 0 2000 4000 6000 8000 Number of Epochs Average Number of Epochs Before Training Terminated 15 seconds of Motor Babbling 15 Hidden Layer Nodes All Available States The Bio-Inspired Set Motor Position and Velocity Only All Motor States Figure 4.14: Average number of epochs used before training was terminated (bars: standard devi- ation). For each sensory set, 25 ANNs (with 15 hidden layer nodes) were trained on 15 seconds of motor babbling. As expected, it took longer to learn when using tendon tension (left two sets), as there were more features to extract from the data. of magnitude better than their tension-less counterparts after 100 epochs. In fact, if we average the training performance for the first ten epochs for each sensory set (Figure 4.13,right), two main observations can be made. The first observation is that ANNs trained on motor information alone will only marginally improve after the 6th epoch and the second is that before the 6th epoch the performance of the ANNs trained with tendon tension information is worse. These observations imply that it is easier to learn the relationship between motor position and velocity information but the relationship is fundamentally incomplete and further training cannot rectify this. Conversely, it takes longer to learn the complex relationship between motors, tendons, and the joints that they actuate, but the performance is drastically better. These results further strengthen the argument that tendon tension information critically enables the accurate prediction of joint angles from non-collocated sensory information. While it is important to understand how an ANN performs on training data, it is perhaps more important to understand how the ANN can generalize to data that is not similar to the data used to train it. To that end we will discuss how these ANNs perform when predicting joint angles 132 for the four generalization movements discussed in Section 4.3.5. It can be seen from the traces in Figures 4.9 & 4.12 or from the bar plots presented in Figure 4.15 that the performance of the ANNs trained with tendon tension information consistently outperform the ANNs trained without it and that this trend is fairly well preserved across the different movement types when comparing average performance. This is more easily seen in Figure 4.15 where, for each sensory set, there is little difference between the average ANN performance across the four different movement types. To explore how these ANNs perform across the movement space we will explore average per- formance as a function of joint angle (Figure 4.16) and as a joint function of joint angle and joint stiffness (Figure 4.17) to better understand how the ANN trained on each sensory set perform across the output space on average. Figure 4.16 represents a radial bar plot where the average performance for joint angles in each Avg. Mean Absolute Error (deg.) 10 -4 10 -3 10 -2 10 -1 10 0 Sinusoidal Angle Sinusoidal Stiffness Point-to-Point Angle Sinusoidal Stiffness Point-to-Point Angle Point-to-Point Stiffness Sinusoidal Angle Point-to-Point Stiffness All Available States Bio-Inspired Set Motor Position and Velocity Only All Motor States Figure 4.15: Bar plots of the average performance of each of the four ANNs (trained on the four sensory sets) when predicting joint angle from the four generalization movements (plotted on a log scale). For each sensory set, 25 ANNs with 15 hidden layer nodes were trained with 15 seconds of motor babbling (1 kHz sampling frequency) and their performance (mean absolute error) was averaged to compare across sensory sets and across movements. For each sensory set, there is little difference across movements, but there is a consistent trend that the sensory sets that include tendon tension (All Available States and the Bio-Inspired Set) perform 3 orders of magnitude better than the sets trained without tendon tension. 133 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 0 -1 -2 -3 -4 Log Average Performance (MAE) vs. Joint Angle Sinusoidal Angle Sinusoidal Stiffness All Available States The Bio-Inspired Set Motor Position and Velocity Only All Motor States Point-to-Point Angle Sinusoidal Stiffness Sinusoidal Angle Point-to-Point Stiffness Point-to-Point Angle Point-to-Point Stiffness Log Avg. MAE (deg) Figure 4.16: Radial bar plots for the log average performance for different joint angle bins (every 15 degrees) for each sensory set across all four generalization movements. The average perfor- mance for each ANN appears to be consistent across the joint angle space (i.e., there is no clear dependence on the actual joint angle and error from the predicted joint angle). While we again see that the ANNs trained with tendon tension information (left two columns) outperform the ANNs trained without it (right two columns) by about 3 orders of magnitude, we now see that this is generally true across the entire joint angle space regardless of movement type. 134 sector (every 15 degrees) are plotted (on a log scale) to identify any dependence of the performance on the expected joint angle for any of the generalization movements. As the average performance for each sensory set do not differ greatly from the average value shown in Figure 4.15 across the joint angle space, we can conclude that there is no clear dependence of performance on the expected joint angle. However, a small deviation from this behavior can be seen in the ANNs that are trained without tendon tension (right two columns) when performing movements where the joint angle is allowed to go towards to edges of the range of motion (second and fourth row) where the performance becomes slightly better at the edges of the range of motion. This can be attributed to the fact that the motor babbling experiments spend more time at the boundaries of the range of motion than they do exploring the interior angles, so it makes sense that these ANNs are “data rich” for these postures resulting in better estimates. These additional data do not appear to affect the performance of the ANNs trained with tendon tension information (left two columns) because their estimates appear to be are accurate everywhere. While it is useful to consider performance as a function of joint angle, these movements also sampled the joint stiffness space. Therefore we similarly plot the average performance as a heat map for different values of joint angle and joint stiffness for each sensory set across all movements (Figure 4.17). It can be clearly seen that the ANNs trained with tendon tension perform equally well across all values of joint angle and joint stiffness across all movements (left two columns). Ad- ditionally, the previous observation made regarding better performance for ANNs trained without tendon tension information (right two columns) at the boundaries of the joint’s range of motion can be seen again here (second and fourth row). What was previously unexplored was how changing the values of joint stiffness affect the performance of these ANNs. We can now see that the ANNs trained without tendon tension appear to perform worse at low values of joint stiffness across all movements (with the set containing Motor Position and Velocity only exhibiting slightly higher errors at these values). This is consistent with the expected results because lower joint stiffness can be attributed to the tendons operating at regions of lower tendon stiffness (called the “toe region” for physiological tendon) where we see more disproportionate lengthening per unit force, which 135 Sinusoidal Angle Sinusoidal Stiffness All Available States The Bio-Inspired Set Motor Position and Velocity Only All Motor States Point-to-Point Angle Sinusoidal Stiffness Sinusoidal Angle Point-to-Point Stiffness Point-to-Point Angle Point-to-Point Stiffness Joint Angle (deg) Joint Stiffness (Nm/rad) 90° -90° 0° 90° -90° 0° 90° -90° 0° 20 30 40 50 20 30 40 50 20 30 40 50 Average Performance (MAE) vs. Joint Angle vs. Joint Stiness 0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 Average MAE (deg.) Figure 4.17: Heatmap of the average mean absolute error versus joint angle and joint stiffness. It is clear that the ANNs trained on tendon tension (left two columns) can reliably predict joint angle at any level of joint stiffness, while the ANNs trained without tendon tension (right two columns) have difficulty at low joint stiffness values (regardless of the movement task). This is because at lower joint stiffness, the tendons are less stiff (i.e., more disproportionate lengthening per unit force) which causes more nonlinear decoupling between motor and kinematic states. 136 causes more nonlinear decoupling between motor and kinematic states. 4.5 Discussion In this Chapter we created a framework for joint angle estimation by which ANNs use limited experience from motor babbling to estimate joint angles. Vertebrate animals apparently seamlessly learn to control redundant tendon-driven limbs with (i) no dedicated joint-angle sensors by (ii) combining different non-collocated sensory information (e.g., muscle fascicle length via muscle spindles and tendon tension via Golgi tendon organs). We therefore explored whether and how ANNs (an analog to trial-and-error learning based on limited experience) can predict joint angles on the basis of proprioceptive data similar to that available in tendon-driven limbs. We find that for different durations of motor babbling (i.e., amount of data) and different ANN architectures (i.e, network structures), the ANNs trained with tendon tension outperform the ANNs trained without it (Figures 4.8–4.12) and that those ANNs trained on the Bio-Inspired Set perform as well (if not better than) the baseline set of all motor and tendon tension states (All Available States). We also conclude that 15 seconds of motor babbling are sufficient to produce consistent results for each of the four sensory sets when used to train ANNs with 15 hidden layer nodes. The per- formance of each ANNs converged near these parameter values when sweeping across both motor babbling duration and the number of hidden layer nodes with small standard deviations in the per- formance between trials. While it was also observed that ANNs trained only on motor information would perform better with only had one hidden layer node, this increase in performance was not large. Comparing the training performance for ANNs with 15 hidden layer nodes trained with 15 seconds of motor babbling also revealed that the ANNs trained on motor information alone (i) needed fewer epochs to converge to their best performance and (ii) reached a performance similar to the final value after as few as 6 epochs. These observations are consistent with the notion that the relationship between actuators and kinematics is easy to learn but that it is fundamentally limited 137 by its lack of tendon tension information. On the computational side, our results reveal that, as expected given the prior chapters, the inclusion of tendon tension improves the estimates of joint angles from those using motor infor- mation alone. But specifically (and perhaps intuitively), we also illustrated that those ANNs that only utilize motor information struggle to predict joint angles the most when the tendon operates in the more nonlinear toe regions of its tension-deformation relationship. This region is associated with larger tendon deformation and therefore larger decoupling between motors and joint angles. Networks trained with tendon tension data, however, did not have any problem generalizing to different motor tasks. The root cause of this improvement in the estimates of joint angles is that the musculotendon (MT) has an internal degree of freedom: MT length alone does not define ac- tuator position (or muscle fascicle length). And tendon tension is a means to estimate this internal degree of freedom. Therefore, we conclude that it is possible to train an ANN on limited non- collocated measurements of motor position, motor velocity, and tendon tension only to reliably estimate joint angles during a variety of movements. Importantly, this novel bio-inspired posture estimation framework with non-collocated sensors in tendon-driven systems (called insideOut) can provide accurate joint angle estimation during dynamical tasks with limited data if tendon tension measurements are available. There are of course limitations, both conceptual and practical, that need to be pointed out. Chief among them is the conceptual quicksand of what ’state’ means in the biological context. In engineering, the concept of state is central to the Newtonian, Lagrangian, Kanesian, and Hamilto- nian approaches to rigid body dynamics and their control: the ‘states of a system‘ are the minimal set of kinematic DOFs (sometimes called generalized coordinates) that suffice to explain the en- ergy transformations the system can undergo, and how to control them. In this context, state, observability, and state estimation are clearly defined. In biology, the question of what is the state and what is state estimation are fraught. In fact, they are best posed from an information-theoretical perspective agnostic to any particular formulation 138 that is likely to anthropomorphize the question. That is why we were careful not to call joint angle the ‘state’ and we do not call our approach ‘state estimation.’ Rather we used the ANN approach as a generic means to answer the information-theoretical question of whether and how well different sets of afferent information can—in principle—estimate joint angle. Therefore, this work has the important consequence of illuminating a glaring gap in spinal neurophysiology: Why do vertebrates have Golgi tendon organs? When discussing the ability of the nervous system to estimate the state of the body for the purpose of control, high-level concepts such as Figure 4.1 are invoked to propose that afferent signals clearly provide such information. But what information do these mechanoreceptors provide and how is it processed? In fact, does the information they provide suffice in principle? The field has been operating under the assumption that, in the absence of direct joint angle sensors, muscle spindles (Mileusnic and Loeb, 2006; Edin and Vallbo, 1990; Ting, 2007) and skin stretch afferents (Edin and Abbs, 1991; Edin, 2001; Hulliger et al., 1979) provide kinematic in- formation which, in principle, should enable estimating and controlling joint angles. Zajac, Loeb, Zahalak, Hatze, Hill, and others who initiated the computational approach to muscle modeling did—from the start—point out that detailed work was needed to understand the interaction be- tween muscle stretch and fascicle length (Zajac, 1989; Loeb, 1984; Zahalak, 1990; Hatze, 1977; Hill, 1953). Muscle spindles, by providing direct estimates of fascicle length and velocity, became the mechanoreceptor of choice because of (i) the availability of heroic experimental recordings from Ia and II afferents in cats and humans around the time these computational approaches came about; and (ii) because those data could be collected in the anesthetized animal or relaxed hu- man. Controlling and measuring tendon tension while recording from Ib afferents in experimental preparations is more difficult. At the time, Loeb conjectured that muscle spindle afferents needed to be informed by Golgi tendon organ afferents to be useful (Scott and Loeb, 1994; Mileusnic and Loeb, 2006). This makes sense for two reasons. First, muscle spindle signals are ‘tuned’ by the fusimotor drive and are a floating, relative measure of fascicle length and velocity. And second, 139 even if correct, the measurements do not account for tendon stretch. This was, in fact, the precedent logic for our hypothesis that the Bio-Inspired Set of afferent data should in principle be better than kinematic data alone. Our results now bring computational rigor to this longstanding question and answer it in the positive: We provide strong evidence that Golgi tendon organs (a kinetic afferent) likely improve kinematic estimates of limb posture. We hope this work catalyzes a new wave of neurophysiological research to continue to close this information- theoretical gap in our understanding of sensorimotor physiology—and how spinal, brainstem and somatosensory neurons process afferent information to enable versatile physical function. 140 Chapter 5 Parameter Sensitivity Analysis: How would changes to the movement task or the plant change these results? 5.1 Abstract How do changes in tendon elasticity or other mechanical parameters affect the minimum amount of sensory information needed to infer kinematic states? This critical question lies at the heart of this chapter. We have established that for a very compliant tendon-driven system, tendon tension information must be included in the minimum set of sensory information used to predict joint angles from a artificial neural network (ANN), increasing the accuracy by nearly 3 orders of magnitude and producing errors on average below 10 3 degrees. But it is critical that we understand how these results extend to a system with different series compliance or perhaps to different movements that may require more of less demanding length changes to be incurred by the tendon. To that end, this chapter will explore three different paradigms to quantify how robust the performance of these ANNs are. First we compare ANNs performance for each sensory set when asked to predict joint angles for sinusoidal movements of different frequencies and conclude that ANNs trained with tendon tension generalize better to higher frequencies. Next, we perform a parameter sensitivity analysis for changes in motor damping and tendon stiffness parameters. We find that ANNs trained with tendon tension lose performance when the tendon stiffness increases, but still outperform those ANNs trained without it. Additionally, we find no relationship between the amount of motor damping and the performance of these ANNs for the range of damping pa- rameters chosen. Lastly, we conduct an experiment on a system with very rigid tendons to see how useful tendon tension information is when tendons do not deform by very much. We find that even for this extreme example, ANNs trained with tendon tension information perform better than 141 those trained without it (even though the difference in performance is much smaller). We conclude that the usefulness of tendon tension is well conserved across different mechanical parameters and across different dynamic constraints. 5.2 Introduction In the previous chapter, we utilized a computational model of a tendon-driven pendulum to explore how observing different sensory states (e.g., motor positions or tendon tensions) affects the performance of artificial neural networks (ANNs) designed to predict joint angle from limited experience motor babbling. We concluded that reasonable predictions can be made from ANNs trained only on motor information (i.e., positions, velocities and/or accelerations), but the inclu- sion of tendon tension information drastically improves the prediction, enabling reliably accurate estimates of joint angles to be made from what was called the Bio-Inspired Set of non-collocated sensory information. While these results are promising and serve as a valuable proof of concept for an algorithm that can predict joint angles in tendon-driven systems using motor positional data and tendon tension measurements (called insideOut), it is important to understand how changes in movement dynamics or changes in the physical characteristics of the plant will affect these results. Therefore, the purpose of the final chapter of this dissertation will be to explore the robustness of these results through parameter sensitivity for tendon stiffness and motor damping parameters and by changing the dynamic constraints of the system (by changing the movement frequency/speed). We will begin by conducting an experiment where the pendulum described in the previous chapter is made to follow a series of sinusoidal movement tasks with the same movement amplitude but variable frequencies. By keeping the joint angles trajectory constant in space and varying the rate at which that trajectory is carried out, we will impose larger dynamic constraints on the system. It is our expectation that because faster movements require larger tendon tensions to overcome larger pendulum torques, tendon deformations will become larger at higher frequencies therefore 142 altering the performance of these ANNs in a frequency dependent way. As such we will show that the performance of ANNs trained on motor information alone will become worse as movements become faster, while those ANNs trained with tendon tension information will not lose much accuracy. Next we will explore the consequences of changes to the mechanical parameters of the plant by varying both motor damping and tendon stiffness. We conclude that for the choices of motor damping used there was no discernible difference in the performance of ANNs trained with tendon tension data. However, there is a slight correlation between the performance of ANNs trained on the set of All Motor States and motor damping. This result is reasonable as larger motor damping will induce larger motor accelerations which may increase the usefulness of a ANN that utilizes this data to predict joint angles. Additionally, we find that increasing tendon stiffness actually decreases the performance of ANNs train with tendon tension while the performance of ANNs trained only on motor information remain relatively unchanged. This is contrary to what we would expect—as tendons become more stiff the tendons deform less so the motor and joint rotations become more coupled, thereby increasing the performance of ANNs trained only on motor information. In a final experiment, we will further explore this relationship between performance and tendon stiffness by choosing parameters that correspond to very high tendon stiffness (near the upper limit of what the feedback linearization algorithm can handle). By calculating how this plant can generalize to novel movements we can finally address how useful tendon tension and motor information becomes when tendons become increasingly rigid. These results align more with our previous intuition where we see an increase in the performance of ANNs trained solely on motor information (i.e., motor behavior! joint behavior in the limit as tendon stiffness!1) and a decrease in the performance of ANNs trained with tendon tension data (i.e., tension becomes less useful when stiffness is too steep). Finally, we conclude that the results of these experiments are fairly robust to changes in move- ment dynamics and mechanical parameters and that more accurate estimates of joint angles can be 143 made when tendon tension information is used. However, we do see that the error associated with using motor information only can be quite manageable when tendon stiffness is sufficiently high, which would suggest that for some applications (where tendon tension data is perhaps too difficult to measure) it is possible to use motor information only to produce a reasonable approximation of joint angle. 5.3 Material and Methods These experiments build off of the methods proposed in the previous chapter. When neces- sary figures and equations have been reproduced, but the reader is directed to those sections for reference. 5.3.1 Sweeping Movement Frequency The goal of this experiment was to see how ANNs trained on the four different sensory sets performed when the frequency of the sinusoidal joint angle movements used to test generalizability increase. In the previous chapter, we derived two movement tasks where the joint angle was varied sinusoidally while the joint stiffness was either covaried sinusoidally or varied with a point-to-point tasks with step durations equal to three times the period of the oscillation. Similarly, in this chapter we will define 8 separate movements (4 joint angle movement frequencies with either sinusoidal or point-to-point joint stiffness tasks) to test how well these ANNs can generalize to faster movements and, by extension, more dynamic or ballistic movement conditions. Equations 5.1–5.2 will define the 8 trajectories forf2f0:5; 1; 2; 4g Hz. 144 Angle Sinusoidal / Stiffness Sinusoidal r j = 4 sin(2ft) K r j = 50 + 20 2 50 20 2 cos(2ft) (5.1a) (5.1b) Angle Sinusoidal / Stiffness Point-to-Point r j = 4 sin(2ft) K r j = 8 > < > : Eq. 4.15 for smooth transitions Otherwise, constant hold phase at random stiffness values for 3=f second minus transition time (0.25 seconds) (5.2a) (5.2b) For each of these movements, the feedback linearization algorithm defined in Section 4.3.6 is used to (i) follow the desired reference trajectory while (ii) recording the associated motor and tendon tension states. These states will be divided into into the four main sensory groups of interest (See Eqs. 4.11–4.14 for reference) and subsequently used to test each ANN’s generalizability at each frequency. We then conduct 50 trials whereby (feedfoward) ANNs with 15 hidden layer nodes are trained on 15 seconds of motor babbling for each sensory set (as done in the previous chapter) and then asked to predict joint angles for each of the 8 generalization movements. The performance of each ANN for each movement is calculated as the mean absolute error and the overall generalizability of each sensory set (for each movement) is calculated as the average across trials. 5.3.2 Sweeping Tendon Stiffness and Motor Damping The next experiment will sweep across both tendon stiffness and motor damping parameters to better understand the affect the these parameters have on the results from the previous chapter. 145 The approach used for each choice of parameter values will be similar to the approach taken in the previous chapter, whereby ANNs for each sensory set will be trained on motor babbling data and their overall performance will be determined by their ability to generalize to different movement tasks. However, whenever the plant is changed, the control needed to produce the generalization trajectories will change too. As a result, the feedback linearization tool must be used for each of the 9 parameter settings described below (3 motor damping and 3 tendon stiffness) in order to generate the sensory sets associated with the reference trajectories derived in Section 4.3.5. Once the generalization trajectories have been created, 50 ANNs (with 15 hidden layer nodes) are built for each sensory set and trained on random motor babbling (15 seconds) to predict joint angle. These ANNs are then asked to predict the joint angle from the generalization trajectory data and their performance is measured by the mean absolute error. The overall generalizability is again calculated as the average performance across all 50 trials for all movements and all sensory sets. Therefore for each of the 9 parameter choices, a single metric is provided for each sensory set which indicates how well each ANN trained on the new plant, allowing us to identify any trends across either parameter. While this is fundamentally the same experiment as previously conducted in Chapter 4 with the only difference being changes to the plant itself, it is important to discuss how changes in the parameters themselves influence the behavior of the plant and why the parameters were chosen in the ranges that they were. As previously discussed, the original choice of tendon stiffness was quite low in order to deter- mine if tendon tension information would be useful for predicting joint angles in a very compliant tendon-driven system. However, to address whether this extreme choice of compliance affects the results, we desired more stiff tendons to compare the results. A fundamental limitation when choosing the tendon stiffness parametersk T andb T came when deciding to compare the same ref- erence trajectories (with the same range of joint stiffness values). At some point, when the tendons become too stiff (for a given choice of motor damping) it is impossible for the system to follow the reference trajectory because the joint stiffness values between [20,50] Nm/rad will be associ- ated with tendon tensions incompatible with the desired joint angle movement. It was determined 146 for the sake of this experiment that it would be best to compare different ANNs which learned on different plants while performing the same task. Therefore, for the range of joint stiffness values chosen for the reference trajectories (defined in Section 4.3.5), we selected tendon stiffness pa- rameters that would exhibited roughly 33% ( 0:0267 m) and 16% ( 0:0125 m) of the original range of tendon deformations ( 0:08 m) seen in the compliant tendon example. As discussed in Section 4.3.2, values of k T and b T were derived by solving the minimum joint stiffness con- straint (2k T b T r 2 j = 10 Nm/rad) and a second constraint that considers when the pendulum is in the vertical position at maximum joint stiffness and the tendons are maximally deformed (i.e., if K T;1 = K T;2 , thenK max j = 2k T b T r 2 j exp(b T l max T;i )). The values derived for this experiment are presented in Table 5.1 and the respective tension-deformation plots are shown in red in Figure 5.1. f T;i ( m;i ; j ) = 8 < : k T exp(b T l T;i ) 1 ; (l T;i 0) 0; (l T;i < 0) (5.3a) (5.3b) where l T;i = r m m;1 r j j ; i = 1 r m m;2 +r j j ; i = 2 andb T > 0; k T > 0 are shape constants. K T;i = @ @l T;i f T;i =k T b T exp(b T l T;i )k T b T (1b T l T;i ) (when l T;i << 1) (5.4) To compare the stiffness of these tendons, we can compare the Young’s modulus for each ten- don when under 400 N of tension (assuming that this tension corresponds to the same percentage of maximum stress, max T;i =f max T;i =CSA, for each tendon). If we consider a cylindrical tendon with diameter (d) of 0.001 m and a slack length (l T;s;i ) between 0.01 and 0.04 m, then Young’s modulus at this stress level (400 N=CSA) can be calculated by equation Eq. 5.5. The approximate ranges ofE for the parameters chosen are provided in the last columns of Table 5.1. 147 E f T;i =400 N = K T;i f T;i =400 N l T;s;i CSA = b T (400 N +k T )l T;s;i CSA (5.5a) (5.5b) Note that the range of Young’s moduli for the stiffness parameters that produce the Medium and High tendon stiffness are 2.6x and 5.2x the range of the default Low stiffness parameters used in the previous experiments, respectively. These higher stiffness values are more consistent with the observed range of Young’s moduli for physiological tendon (1.2–1.7 GPa), allowing observations to be made about the utility of physiological tendon sensors (i.e., Golgi tendon organs) and their role in sensory fusion (Bennett et al., 1986; Zajac, 1989; Pollock and Shadwick, 2017). Table 5.1: List of parameters chosen to describe tendon tension-deformation relationship (defined in Section 4.3.2 and reproduced in Eq. 5.4) and the relative stiffness (compared to the default value used in Chapter 4) for the experiments described in Sections 5.3.2 (Parameter Sweep) & 5.3.3 (Very High Stiffness). Note that in order to make the minimum joint stiffness consistent across trials the valuek T b T (i.e., the minimum tendon stiffness given by Eq. 5.4) was conserved for each choice of tendon stiffness parameters. (*) The approximate range of Young’s moduli values (E) were calculated from a conservative 0.001 m wide tendon with resting lengths between 0.01 and 0.04 m when under 400 N of tension. Experiment Tendon Stiffness k T b T Approx.E Range (GPa) Parameter Sweep (Section 5.3.2) Low (default) 100 20 [0.127, 0.509] Medium 33.3 60 [0.331, 1.324] High 16 125 [0.662, 2.648] Very High Stiffness (Section 5.3.3) Very High 2 1000 [5.118, 20.474] 148 0 2 4 6 8 Tendon Deformation (cm) 0 100 200 300 400 Tendon Force (N) k T = 33.3 b T = 60 k T = 16 b T = 125 k T = 2 b T = 1000 k T = 100 b T = 20 Very High Stiffness Experiment Parameter Sweep Experiment Figure 5.1: Examples of tendon tension deformation curves for the parameters chosen for the Parameter Sweep Experiment (red, Section 5.3.2) and the Very High Stiffness Experiment (blue, Section 5.3.3). The trace seen in solid red represents the force-length relationship for the compliant tendons used in Chapter 4. Choosing different values for the motor damping parameter (b m ) has large effects on the plant (and its simulation) behavior and therefore limited the range of values explored and required care- ful consideration. For motor damping values that were too low, simulations of contact at the boundaries became unstable when the pendulum was allowed to more rapidly flip back from one boundary to the other. As discussed at length in the previous chapter, the way in which contact forces are handled in this simulation is by allowing the pendulum to “enter into” the boundary for a single time step before being instantaneously moved back to the boundary while applying a restorative force to the pendulum (equal and opposite to the applied torque) to prevent it from going further into the boundary (i.e., any attempts to move into the boundaries will force the pen- 149 dulum to remain at the boundary). When the contacts were this rapid and the motors were allowed to rotate more freely with less damping, the boundaries became very unstable and the simulation broke. To account for this we ran the simulation at 50 times the sampling frequency (decreasing the time step by a factor of 50) to avoid large contacts at the boundaries and then proceeded to down-sample that data by a factor of 50 once the simulation was complete so that the ANNs would still be training on the same duration (and amount) of sensory data. At the other end of the spectrum however, making the motor damping too large came with its own complications. Because of the way in which we have defined motor babbling to be slightly coupled by forcing some level of cocontraction, when motor damping is too large the pendulum is incapable of moving from one side of the configuration to the other resulting in poor motor babbling data. This is because larger values of motor damping effectively filter the activation signals and remove what little difference they had been prescribed. Therefore, the tensions on the tendons produced by both motors are closer together in magnitude and will not be able to produce a net torque sufficient to overcome gravity. Of course it is possible to remove gravity for this simulation and then any net difference in tendon tensions would result in movement, but as it is our intention to compare physical plants, we instead chose a value that allowed us to still conduct useful motor babbling trials. Therefore the motor babbling values of interest were chosen to be 0.5x, 1x, and 2x the nominal value provided in Table 4.1 (0.00462 Ns/m). 5.3.3 Very High Tendon Stiffness Experiment As we were limited in how stiff we could make the tendons while still maintaining the reference joint angle trajectory and a joint stiffness range of [20,50] Nm/rad, a separate experiment was con- ducted where the tendon stiffness is chosen near the upper limit of what is allowed by a feedback linearization controller and new generalization trajectories were generated from a more reasonable joint stiffness range for this choice of tendon stiffness. This allowed us to answer the question, 150 “What happens when tendon stiffness approaches infinity and the tendons become rigid?” To do this, we began by choosing a set of tendon stiffness parameters (k T andb T ) that would describe a tendon that deforms roughly a tenth as much as the Low stiffness example used in Chapter 4 and Section 5.3.2 while still conserving the minimum stiffness value (k T b T ). We found that choosing k T = 2 andb T = 1000 satisfied these conditions, resulting at a deformation of roughly 5.3 mil- limeters when exposed to 400 N of tension. If we consider the dimensions of tendon described in the Section 5.3.2, we can approximate the range of Young’s modulus for this tendon to be between 5.118 and 20.474 GPa (Table 5.1). When conducting a preliminary motor babbling experiment with these new tendon stiffness parameters, it was observed that the range of induced joint stiffness values was quite large with a maximum of around 2000 Nm/rad. Therefore, the reference trajectories that will be used to test ANN generalizability should occupy a portion of this range. It was determined that keeping the joint stiffness between 150 and 650 Nm/rad for the four generalization trajectories would (i) better represent the typical joint stiffness values the plant will experience during babbling while (ii) allowing the feedback linearization algorithm to adequately control these trajectories. As we did for the previous experiments, generalization trajectories are generated using the feedback linearization controller (See Section 4.3.6) and th resulting sensory information is divided into the four sensory sets. Then 50 ANNs were generated for each sensory set (15 hidden layer nodes) and trained on random babbling data (15 seconds) to predict joint angle. The ability of each ANN to generalize was measured as the mean absolute error for each of the four movement trajectories and the average performance was then calculated to identify how these performances compare to the considerably lower stiffness examples seen in previous experiments. 151 5.4 Results 5.4.1 Sweeping Movement Frequency Results from the previous chapter showed that ANNs trained with tendon tension information (i.e., the Bio-Inspired Set and the set of All Available States) outperform those ANNs that trained without it (i.e., the sets of Motor Position and Velocity Only and All Motor States) when predicting joint angles from a subset of generalization movements that sample the joint angle and joint stiff- ness task space. While these results provided evidence to support the notion that tendon tension measurements critically enables accurate joint angle estimation (in both biological and artificial neural networks), it was limited in its scope by neglecting the affect that changes in movement constraints like speed have on the results. It is expected that as a movement becomes more fre- quent or ballistic, that larger tendon tensions (and therefore tendon deformation) will be needed to accelerate and brake the movement appropriately. Therefore we expect that (i) the performance of the ANNs trained only on motor information will generalize poorly to these more demanding, more rapid movements and (ii) the ANNs that train with tendon tension information should gen- eralize well as they can correct for large changes in tendon behavior responsible for decoupling motor and joint angle states. Figure 5.2 demonstrates the average performance of 50 ANNs for each sensory set when asked to generalize to sinusoidal joint angle tasks with frequencies of 0.5, 1, 2, and 4 Hz (where joint stiffness is either covaried sinusoidally or varied in a point-to-point task, Eqs. 5.1–5.2). The re- sults here are consistent with our expectations, where we see that the ANNs trained only on motor information will see average errors increase from 10 0 to 10 1 degrees as the frequency of the joint angle trajectory increases (regardless of the joint stiffness trajectory), while the performance of the ANNs trained with tendon tension information will only decrease from 10 3 to 10 2 . Note that while the error for all ANNs will increase by one order of magnitude as the frequency 152 Sinusoidal Angle Point-to-Point Stiffness Sinusoidal Angle Sinusoidal Stiffness Avg. Mean Absolute Error (deg.) 10 -3 10 -2 10 -1 10 0 0.5 1 2 4 0.5 1 2 4 0.5 1 2 4 0.5 1 2 4 10 1 Movement Frequency (Hz) All Available States The Bio-Inspired Set Motor Position and Velocity Only All Motor States Figure 5.2: Bar plot of the average performance (MAE) of each sensory set as function of the fre- quency of the sinusoidal joint angle trajectory. The joint stiffness was either varied sinusoidally or by a point-to-point task (Eqs. 5.1–5.2). The ANNs trained on tendon tension (left two sets) appear to generalize better to higher frequency movements, only worsening slightly when the movements becomes fastest. The ANNs trained on motor information only may decrease their performance by a similar order of magnitude, but there is quite a difference between producing average errors of 10 2 and 10 1 degrees. increases, the performance of ANNs trained with tendon tension are still very reasonable at high frequencies, but the errors incurred from estimating joint angles from motor information alone be- come quite large (suggesting ANNs trained without tendon tension information could not reliably be used in practice). It is, however, interesting that the performance of ANNs trained with tendon tension informa- tion decreased as the movements became faster (even if only slightly) as the working hypothesis is that tendon tension information is used in these ANNs to reconcile the tension-specific deforma- tion that decouples the motor angles from the joint angle, providing useful estimates at any tendon tension. To explore this further, for each movement task and each sensory set, the movement was divided into joint angle bins (every 15 degrees) and the average performance was calculated for each bins to see if errors in joint angle estimation have any dependence on joint angle itself when the frequency is increased. The results across frequencies were plotted for each sensory set as 153 radial bar plots in Figures 5.3 and 5.4 for the movements where joint stiffness was varied sinu- soidally or in a point-to-point task, respectively. First, it is important to note that regardless of the choice of joint stiffness reference trajectory, the ANNs trained on motor information only (bottom row for both figures) are relatively consistent across the joint angle space (implying that the perfor- mance of these ANNs does not depend on the joint angle itself for this task—it is consistently poor everywhere). Secondly, and perhaps more interestingly, the performance for ANNs trained with tendon tension information (top row for both figures) appears to be consistent for lower frequency movements, but develops some asymmetry when the movement becomes increasingly fast. It can be seen in both figures that the performance of these ANNs gets worse at high frequencies when the pendulum approaches the boundaries of the oscillations. This apparent speed-accuracy trade off has important consequences to the observed physiological phenomenon of the same name. That is, if the nervous system were to somehow utilize tendon tension information from the Golgi ten- don organs to better estimate posture and subsequently use this internal model for control, when point-to-point tasks becomes too fast, the internal representation begins to fail near the movement amplitude which could account for poor accuracy in the controller. In the robotic analogue, a simi- lar consequence exists where accuracy at the target amplitude of a movement could decrease when movements become too fast as the controller may not be able to stably track the movement. 5.4.2 Sweeping Plant Parameters Another limitation of the experiments from the previous chapter stems from the use of a fixed computational model of the mechanical plant. In order to address the usefulness of this algorithm, it is therefore necessary to perform parameter sensitivity on the model to compare the how the re- sults change with changes to the plant. As discussed previously, 9 separate models were made from 3 selections of tendon stiffness parameters and 3 selections of motor damping. For each plant, 50 ANNs were generated for each of the sensory sets, trained on motor babbling, and asked to predict joint angles for the same four generalization trajectories (identical in joint angle and stiffness tra- 154 10 -3 10 -1 10 1 10 -3 10 -1 10 1 10 -3 10 -1 10 1 Average Performance (Avg. MAE in deg.) 10 -3 10 -1 10 1 All Available States 4 Hz 2 Hz 1 Hz 0.5 Hz The Bio-Inspired Set 4 Hz 2 Hz 1 Hz 0.5 Hz Motor Position and Velocity Only 4 Hz 2 Hz 1 Hz 0.5 Hz All Motor States 4 Hz 2 Hz 1 Hz 0.5 Hz Sinusoidal Angle / Sinusoidal Stiffness Figure 5.3: Radial bar plot of the average performance (MAE) of each sensory set as function of both the frequency of the sinusoidal joint angle trajectory and joint angle when the joint stiff- ness is varied sinusoidally as well (twice the frequency). The ANNs trained on tendon tension (top two sets) generalize better to higher frequency movements, only worsening slightly when the movements becomes fastest. For these two sets, when movements are the fastest, the largest errors appear to occur at the boundaries of the sinusoidal movement (which has interesting consequences to speed/accuracy trade offs). 155 10 -3 10 -1 10 1 10 -3 10 -1 10 1 10 -3 10 -1 10 1 Average Performance (Avg. MAE in deg.) 10 -3 10 -1 10 1 All Available States 4 Hz 2 Hz 1 Hz 0.5 Hz The Bio-Inspired Set 4 Hz 2 Hz 1 Hz 0.5 Hz Motor Position and Velocity Only 4 Hz 2 Hz 1 Hz 0.5 Hz All Motor States 4 Hz 2 Hz 1 Hz 0.5 Hz Sinusoidal Angle / Point-to-Point Stiffness Figure 5.4: Radial bar plot of the average performance (MAE) of each sensory set as function of both the frequency of the sinusoidal joint angle trajectory and joint angle when the joint stiffness is varied with a point-to-point task. Again, ANNs on tendon tension (top two sets) generalize better to higher frequency movements, only worsening slightly when the movements becomes fastest. For these two sets, when movements are the fastest, the largest errors appear to occur at the boundaries of the sinusoidal movement (which has interesting consequences to speed/accuracy trade offs). 156 jectories, not in the motor and tension states that produce it; see Section 5.3.2). Therefore, for each choice of tendon stiffness and motor babbling parameters, the generalizability of the four sensory sets was calculated as the average performance for each movement task. Figure 5.5 plots all 9 of these data point for each sensory set and each movement task as a func- tion of tendon stiffness (designated Low, Medium, and High). It can be seen that the performance of ANNs trained on tendon tension information actually decreases as the tendons become more rigid, but still outperform the ANNs trained only on motor information by roughly 2 orders of magni- tude. These results are counter-intuitive as we expected the performance of the ANNs trained on motor information to improve as the tendons became more rigid as the decoupling between motor and joint states would decrease. These results indicate (i) that tendon tension information becomes less useful (but not useless) when tendon stiffness increases, causing the ANNs performance to degrade from training on data that is redundant and (ii) that these tendon stiffness values are still largely nonlinear at the range of joint stiffness values chosen for the reference trajectories (both of these will be addressed in the next experiment). To better observe any affects that motor babbling has on performance, a similar plot was gener- ated where all 9 of the performance values for each sensory set and each movement task are plotted as a function of motor damping (designated Low, Medium, and High). An interesting consequence to viewing the data from this perspective is that we see another clear separation of the performance of ANNs trained with tendon tension information when the tendon stiffness is increased (i.e., the separation of the horizontal lines in the plots of the left two columns). There does not appear to be any clear trends, however, when comparing performance and the amount of motor damping used. There may be a slight positive trend (negative slope) between the performance of the ANNs trained on the All Motor States and the amount of motor damping, as can be seen in the last column of Fig- ure 5.6, but it is not significant. Similarly, it could be argued that the same trend exists for ANNs trained on the set of All Available States when the tendons are compliant (i.e., Low stiffness), but disappears when the stiffness increases. Interestingly (but not surprisingly), these sensory sets are 157 Average Performance (MAE) vs. Tendon Stiness 10 -4 10 -3 10 -2 10 -1 10 0 10 1 10 -4 10 -3 10 -2 10 -1 10 0 10 1 10 -4 10 -3 10 -2 10 -1 10 0 10 1 Low High Medium Tendon Stiness Average MAE (deg.) Low High Medium Low High Medium Low High Medium 10 -4 10 -3 10 -2 10 -1 10 0 10 1 Sinusoidal Angle Sinusoidal Stiffness All Available States The Bio-Inspired Set Motor Position and Velocity Only All Motor States Point-to-Point Angle Sinusoidal Stiffness Sinusoidal Angle Point-to-Point Stiffness Point-to-Point Angle Point-to-Point Stiffness Motor Damping High Medium Low Figure 5.5: Comparing the average performance of ANNs designed to predict joint angles from one of four sensory sets when tendon stiffness parameters are varied. Changes in motor damping can also be seen here and are denoted by line styles in the legend. It is interesting to note that the performance of ANNs trained with tendon tension information (left two columns) perform worse as tendons become more rigid, while the performance of ANNs trained with motor information only (right two columns) do not appear to be affected by changes in tendon stiffness. 158 Average Performance (MAE) vs. Motor Damping 10 -4 10 -3 10 -2 10 -1 10 0 10 1 10 -4 10 -3 10 -2 10 -1 10 0 10 1 Low High Medium 10 -4 10 -3 10 -2 10 -1 10 0 10 1 Motor Damping Average MAE (deg.) Low High Medium Low High Medium Low High Medium 10 -4 10 -3 10 -2 10 -1 10 0 10 1 Sinusoidal Angle Sinusoidal Stiffness All Available States The Bio-Inspired Set Motor Position and Velocity Only All Motor States Point-to-Point Angle Sinusoidal Stiffness Sinusoidal Angle Point-to-Point Stiffness Point-to-Point Angle Point-to-Point Stiffness Tendon Stiffness High Medium Low Figure 5.6: Comparing the average performance of ANNs designed to predict joint angles from one of four sensory sets when motor damping is varied. Changes in tendon stiffness can also be seen here and are denoted by line styles in the legend. We again see from the vertical separation of the lines in the left two columns that ANNs trained with tendon tension information perform worse as tendons become more rigid—a trend not observed in the ANNs trained with motor information only (right two columns). Additionally, as seen in Figure 5.5, there appears to be no trends in performance with respect to motor damping except for those ANNs trained on all motor states (including acceleration, right column), which intuitively makes sense as higher damping may mean more useful information in the motor acceleration states. 159 Log Average Performance (MAE) vs. Tendon Stiness and Motor Damping Motor Damping Tendon Stiness Sinusoidal Angle Sinusoidal Stiffness All Available States The Bio-Inspired Set Motor Position and Velocity Only All Motor States Point-to-Point Angle Sinusoidal Stiffness Sinusoidal Angle Point-to-Point Stiffness Point-to-Point Angle Point-to-Point Stiffness -2.9 -3.0 -3.4 -2.4 -2.5 -2.3 -2.0 -1.9 -1.6 -3.4 -3.4 -3.2 -2.9 -2.9 -2.9 -2.0 -2.1 -2.0 -0.0 +0.0 +0.1 +0.1 +0.0 +0.1 +0.0 +0.1 +0.1 -0.0 -0.1 -0.1 +0.0 -0.1 -0.2 -0.0 -0.1 -0.1 -2.9 -3.1 -3.4 -2.4 -2.5 -2.3 -1.9 -1.9 -1.6 -3.4 -3.4 -3.3 -2.9 -2.9 -2.9 -1.9 -1.9 -1.9 -0.0 -0.0 -0.0 -0.0 -0.0 -0.1 -0.1 -0.0 -0.0 -0.1 -0.2 -0.1 -0.1 -0.1 -0.2 -0.1 -0.1 -0.2 -2.9 -3.1 -3.4 -2.5 -2.6 -2.3 -2.0 -1.9 -1.6 -3.4 -3.4 -3.3 -3.0 -2.9 -2.9 -2.0 -2.1 -1.9 +0.0 -0.0 +0.1 +0.1 +0.1 +0.0 +0.0 +0.1 +0.1 -0.0 -0.1 -0.1 +0.0 -0.1 -0.2 +0.0 -0.1 -0.1 -3.0 -3.1 -3.4 -2.5 -2.6 -2.3 -2.0 -2.0 -1.6 -3.5 -3.5 -3.3 -3.0 -2.9 -2.9 -2.0 -2.0 -1.9 -0.0 -0.0 +0.0 +0.0 +0.0 +0.0 -0.0 -0.0 -0.0 -0.1 -0.2 -0.2 -0.0 -0.1 -0.2 -0.0 -0.1 -0.2 Low High Medium Low High Medium Figure 5.7: Heatmap representation of the relationship between tendon stiffness, motor damping, and the performance of ANNs that utilize one of four sensory sets to predict joint angle from limited-experience babbling data. The logarithm of the performance (MAE) has been placed in each square where negative values correspond to good performance. Note that, as seen in Figures 5.5 & 5.6, the ANNs trained with tendon tension information (left two columns) have worse perfor- mance at higher tendon stiffnesses while the ANNs trained on motor information do not show such a trend. Additionally, we can see a slight correlation between motor damping and the performance of ANNs trained with all motor states (including acceleration, left columnn), but no identifiable trends for the other sets. 160 the only ones that include motor acceleration, and it would be expected that higher motor damp- ing would cause more useful motor acceleration information leading to better performance in the ANNs that utilize it. This trend may only be seen in the low tendon stiffness case for the set of All Available States because the decrease in the usefulness of tendon tension information at higher ten- don stiffness values (discussed above and seen in Figure 5.5) may outweigh this marginal increase in the usefulness of motor acceleration information. The combined data from these two figures is additionally presented as a heat map in Figure 5.7 where it these trends can be more clearly seen. 5.4.3 Very High Tendon Stiffness Experiment This last experiment tested the extreme case of very stiff tendons to see how the performance of ANNs trained on these four sensory sets changed. Figure 5.8 demonstrates the average perfor- mance of 50 ANNs trained for each sensory set when generalizing to the novel movement tasks with joint stiffness ranges that are appropriate for these very stiff tendons. It is clear that the per- formance of ANNs trained with tendon tension information still outperform those ANNs trained without it but the difference in performance is much smaller. These results are also consistent with the observations made in the previous experiment, where the performance of ANNs trained with tendon tension information decreases when the tendons becomes increasingly stiff, and yet they still outperform the ANNs trained with motor information only (i.e., tendon tension information becomes less useful and redundant when tendons become more rigid, but not useless!). Addition- ally, we see a large increase in the utility of motor information as the ANNs trained without tendon tension increased their performance from 10 0 to 10 1 , consistent with the original hypothesis that ANNs that trained on systems with more rigid tendons should see an increase in their per- formance because the behavior of the motors and the joint should be more coupled. This sudden increase in motor information utility may be largely due to changes in the range of joint stiffness values used to generate the generalization movements. More specifically, higher joint stiffness ranges correspond to smaller tendon deformations per unit force applied to the tendon and there- 161 Avg. Mean Absolute Error (deg.) 10 -4 10 -3 10 -2 10 -1 10 0 Sinusoidal Angle Sinusoidal Stiffness Point-to-Point Angle Sinusoidal Stiffness Point-to-Point Angle Point-to-Point Stiffness Sinusoidal Angle Point-to-Point Stiffness All Available States Bio-Inspired Set Motor Position and Velocity Only All Motor States Figure 5.8: Bar plots of the average performance for ANNs for each sensory set when the tendon stiffness values are very high (See Section 5.3.3 for explanation). We find that ANNs trained with tendon tension information (the Bio-Inspired Set and the set of All Available States), still outper- form those ANNs trained on motor information alone (the sets of Motor Position and Velocity Only and All Motor States). However, the difference is not nearly as large as in the previous experiments (3 orders of magnitude) because (i) the ANNs trained on motor information only improved their performance by nearly one order of magnitude and (ii) the ANNs that train with tendon tension information continue the trend of worsening performance when tendon stiffness increases (increas- ing errors by nearly 1 order of magnitude). fore higher coupling between motor and joint behaviors which would allow for better joint angle predictions to be made from ANNs that only use motor information. Comparing the training performance after each epoch and the average number of epochs needed to train each ANN helps support the claims that tendon tension information becomes less useful while motor information becomes more useful when tendons becomes very stiff. In Figure 5.9 (right), the average number of epochs for each sensory set is plotted in a bar graph. Comparing these results to the previous experiment where tendons were very compliant (Figure 4.14 for refer- ence) reveals that (i) training ANNs only on motor information requires nearly double the number of epochs (i.e., more information requires more training to extract the relationship) and (ii) the ANNs that train on tendon tension data require fewer epochs on average the previous experiment (i.e., less information can be extracted and a minimum is reached sooner). This can also be seen 162 0 2k 1k 3k 4k 5k Average Number of Epochs Before Training Terminated Training Performance (RMSE in deg.) vs. Epoch Number 0 2k 4k 6k 8k 10k 10 -4 10 -3 10 -2 10 -1 10 0 10 1 10 2 All Available States Bio-Inspired Set Motor Position and Velocity Only All Motor States Figure 5.9: The training performance versus epoch number for 50 trials for each sensory set over- laid to show trends (left) and the average number of epochs used before terminating training (right). Compared to Figure 4.13, the training does not appear to be different for the ANNs trained with ten- don tension information (with similar convergence rate, final performance, and number of epochs used), but the training of ANNs that use the Motor Position and Velocity Only or All Motor States sets now have longer training periods (i.e., more epochs) and better performance. This is consistent with the notion that as the tendons become more rigid, the motor states becomes more useful and longer training periods are needed to extract that information. when the training performance after each epoch for each ANN are overlapped on the same plot (Figure 5.9, left). However, even though the motor information is becoming more useful, the addi- tional information provided by the tendons still clearly produces better training performance (i.e., tendon tension may become less useful, but it is not useless!). Figure 5.10 demonstrates the average performance of ANNs trained on each sensory set across the new generalization movements with respect to joint angle by calculating the average perfor- mance of angular bins (every 15 degrees) and plotting them as radial bar plots. Other than the performance of ANNs trained on either the set of All Available States or the Bio-Inspired Set when performing the task where both joint angle and joint stiffness are varied sinusoidally (first row), there does not seem to be any dependence on the performance of these ANNs and the joint angles they are predicting. The bizarre decrease in performance near the vertical pendulum position for these two ANNs is largely due to the fact that these ANNs have trouble generalizing at lower stiff- 163 Log Average Performance (MAE) vs. Joint Angle -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 -2 -1 -3 Sinusoidal Angle Sinusoidal Stiffness All Available States The Bio-Inspired Set Motor Position and Velocity Only All Motor States Point-to-Point Angle Sinusoidal Stiffness Sinusoidal Angle Point-to-Point Stiffness Point-to-Point Angle Point-to-Point Stiffness Log Avg. MAE (deg) Figure 5.10: Polar bar plots to demonstrate the performance of ANNs trained on each sensory set when generalizing to different movements as a function of joint angle. Most of the performance values are consistent across the joint angle space with the exception of the task where both the joint angle and joint stiffness were varied sinusoidally (with joint stiffness at twice the frequency to have maximum stiffness at the boundaries of the movement). This would indicate that these ANNs are making better use of the tendon tension information at higher stiffnesses (and performing worse at lower stiffnesses where the nonlinearity of the tendon-tension deformation curve becomes more disporportionate). 164 0.2 0 Avg. MAE (deg) Sinusoidal Angle Sinusoidal Stiffness All Available States The Bio-Inspired Set Motor Position and Velocity Only All Motor States Point-to-Point Angle Sinusoidal Stiffness Sinusoidal Angle Point-to-Point Stiffness Point-to-Point Angle Point-to-Point Stiffness Joint Angle (deg) Joint Stiffness (Nm/rad) 90° -90° 0° 90° -90° 0° 90° -90° 0° 150 650 Average Performance vs. Joint Angle vs. Joint Stiness 150 650 150 650 0.1 0 Avg. MAE (deg) 0.2 0 Avg. MAE (deg) 0.4 0 Avg. MAE (deg) Figure 5.11: Average performance heat maps for each sensory set and each generalization move- ment as a function of both joint angle and joint stiffness. As it can be seen, the ANNs trained on tendon tension data (left two columns) still outperform the ANNs trained only on motor informa- tion (right two columns) and appears to better generalize at (i) lower stiffness values and (ii) joint angles that neither correspond to the vertical position or the boundaries of the range of motion. 165 ness values and this movement by design has minimal stiffness when crossing the vertical position. Figure 5.11 evaluates bins of both joint angle and joint stiffness and represents the average per- formance of the bins as a heat map for each sensory set and each movement task. While the errors for the ANNs trained only on motor information are roughly an order of magnitude less than the compliant tendon example presented in Chapter 4, the overall trends are still the same. Intuitively, the ANNs trained only on motor information have the most difficulty when joint stiffness is low (i.e., tendon stiffness is low and tendon deformation is more disproportionate per unit force). The ANNs trained with tendon tension data do not appear to have this difficulty (or at least not to this extent, as we know from the first row of Figure 5.10 that marginally higher errors occur at lower joint stiffness values). 5.5 Discussion The goal of this experiment was to provide parameter sensitivity analysis to support the notion that observing tendon tension information critically enables reliably accurate joint angle estimation by ANNs that train on non-collocated sensory data. The experiments from the previous chapters highlighted the importance of tendon tension; this often forgotten variable that holds invaluable information about joint stiffness, muscle/actuator behavior, and control constraints. The ultimate goal for this chapter, and this dissertation, was to flip this problem upside down or, better yet, insideOut to explore how useful tendon tension information could be when used to predict the joint angle of a compliant tendon-driven system. And while the previous chapter laid the foundation for this work, revealing that ANNs that want to predict joint angles from sensory measurements are drastically improved by the inclusion of tendon tension information, the question still remained, “How well does this algorithm generalize across (i) different dynamic movements and (ii) different plants?” Therefore, this chapter naturally serves as an extension of this previous work where we systematically explore changes to movement constraints (e.g., slow versus ballistic movements) as 166 well as changes to the mechanical parameters that would likely change the results of a system that relies on motor and tendon behavior (i.e., motor damping and tendon stiffness, respectively). We began by expanding upon the previous chapter’s experimental paradigm by training 50 ANNs for each sensory set (with 15 hidden layer nodes and 15 seconds of motor babbling) and then asked them to predict the joint angle for various sinusoidal movements where the frequency was varied between 0.5 and 4 Hz. We found that the ANNs that train with tendon tension infor- mation in addition to motor information (e.g., the Bio-Inspired Set of sensory states) generalize better to faster movements than ANNs that train without tendon tension data, producing average prediction errors as large as 10 2 degree. Interestingly, it was also observed that for high frequency movements, the performance of ANNs trained with tendon tension information decreased at the target amplitude of the oscillation; exhibiting what appears to be a speed accuracy trade-off. This type of trade-off has been well established in humans where faster movements come at the cost of lower accuracy at the movement target (Fitts, 1954). If there exists a similar neural network in biological systems that utilizes tendon tension information from the Golgi tendon organs as well as muscle fascicle length and velocity information from muscle spindles to predict joint angles and these estimates are subsequently used for control, then these results present an interesting inter- pretation of Fitts’ law, whereby poor accuracy is the result of poor state estimation. Regardless, these results indicate that ANNs trained on both motor position and tendon tension information are robust to changes in movement speed. Next, the effect of changes in motor damping and tendon stiffness were simultaneously ex- plored. We concluded that increasing tendon stiffness naturally decreases the usefulness of tendon tension information because the amount of tendon deformation per unit force is lower. We did not see a significant increase in performance when motor damping was increase, although a slight in- crease was observed in ANNs trained with motor acceleration (i.e., more motor damping increases the usefulness of motor acceleration data). Lastly, we tested an extreme case where the tendons were very stiff. We find that even in this 167 rigid example, including tendon tension information increases the accuracy of the prediction. It is important to mention however, that in the case where tendon stiffness increases substantially, while the performance of ANNs trained with tendon tension is better than the performance of ANNs trained without it, the ANNs trained only on motor information now only produce errors in the sub-degree range. It is therefore possible, that for tendon-driven systems with rigid tendons, it may not be necessary to observe the hard-to-measure tendon tension information as predictions made from motor information are sufficient. In the end, we conclude that regardless of the tendon architecture, the actuator behavior, or the movement complexity, tendon tension information is useful (if not at times vital) when predicting joint angles from non-collocated sensory signals. 168 Conclusion and Future Work This thesis was broken down as into two main sections; (1) the exploration of tendon-driven control and the role that tendon elasticity plays in it and (2) the exploitation of this role to bet- ter estimate posture from ANNs trained with tendon tension information. In the first section, we illustrated that approximating muscle/actuator mechanics from joint kinematics does not capture the true behavior of a tendon-driven system nor the relationship between kinematic states and actuator states. In Chapter 1, we first explored how kinematic redundancy inherently produces differences in musculotendon (MT) behavior even for similar movements. But as this system ex- cluded tendon elasticity in its analysis, we were compelled to extend this research to tendon-driven systems with compliant tendons. Therefore, in Chapter 2 a simple, redundant tendon-driven sys- tem (i.e., “tug of war” on a pendulum) was simulated and controlled via an integrator backstepping algorithm to systematically explore the relationship between muscle/actuators and elastic tendons. This approach highlighted that even for identical movements, the internal states of the actuators (e.g., muscle fascicle lengths and velocities) will differ from the kinematically-derived MT states in a nonlinear, tendon tension dependent fashion. This research naturally led to the central ques- tion asked and answered in Chapter 3; can we explicitly derive the amount by which changes in muscle fascicle lengths deviate from MT excursions (i.e., when and how should muscle fascicle lengths be approximated from the kinematics?). A more accurate calculation of muscle fascicle lengths is therein derived to account for MT excursions (limb kinematics) and tendon deformation (kinetics)—exposing an explicit relationship between muscle (actuator) states, tendon tension, and posture. We conclude from the research conducted thus far that (i) kinematics alone are not enough to capture the true behavior of muscle fascicles but instead (ii) kinematics and tendon tensions (i.e., kinetics) complete the picture—allowing for a more accurate equation of muscle fascicle lengths to be derived—implying that (iii) there exists a functional relationship between muscle lengths, tendon tensions, and posture. 169 In the second half of this dissertation, we exploited this relationship to design ANNs that can predict joint angles from motor and/or tendon tension information and limited training. In Chapter 4, we train ANNs with different amounts (i.e., duration of motor babbling) and types (i.e., sensory sets) of information and different network structures to determine what sensory information is crit- ical for joint angle estimation and find that the inclusion of tendon tension information drastically improves the joint angle estimate when compared to those made with actuator states alone. We additionally found that those ANNs trained with tendon tension information could generalize bet- ter to different unlearned movements. Finally, in Chapter 5, we perform parameter sensitivity and find that these results are robust to changes in tendon stiffness level and motor damping as well as across different movement constraints (from slow to rapid). We conclude that the sufficient set of sensory information needed to predict joint angles in a tendon-driven system contained motor positions/velocities and tendon tensions (called the Bio-Inspired Set), and ANNs trained on this set of information produce reliably and robustly accurate estimates of joint angles. The research described here additionally facilitates future work to be conducted where this insideOut algorithm for utilizing bio-inspired sensor information to predict joint posture in tendon-driven systems can be extended to multi-joint, multi-articulating systems in both simulation and in hardware. Ad- ditionally, the accuracy of these networks appears to be related to joint stiffness with intuitively higher errors at lower joint stiffness values which are known to be associated with the more nonlin- ear regions of the tension-deformation curves of the tendons. It is therefore possible to extend this research further in the future to design a high-level controller to work in tandem with the insideOut algorithm to control for higher stiffness values when more accurate estimates of posture are needed (e.g., in a precision task) or for lower stiffness values when less accuracy is acceptable (e.g., during the transition between points). By extension, these results also provide strong evidence that, for biological tendon-driven sys- tems, incorporating Golgi tendon organs information (i.e., tendon tension) with muscle spindle in- formation (i.e., muscle fascicle lengths and velocities) likely improves kinematic estimates of limb posture. This biological implication enables a new wave of neurophysiological research to continue 170 to close this information-theoretical gap in our understanding of sensorimotor physiology—and how spinal, brainstem and somatosensory neurons process afferent information to enable versatile physical function. In the end, one fundamental truth can be drawn from this research: Tendon tension should never be overlooked.... It demands careful scrutiny. 171 Glossary actuator The component that receives input and drives the system dynamics. For biological tendon-driven systems, this is the muscle, while for robotic tendon-driven systems this is a motor. xiii, xiv, xxvi, 1, 2, 3, 5, 7, 32, 35, 37, 38, 49, 52, 57, 94, 96, 97, 99, 100, 137, 166, 168, 169 actuator state A state that describe the actuator’s position or velocity (or acceleration). For mus- cle this corresponds to length and velocity, while for motors this corresponds to motor angle and angular velocity. 6, 7, 94, 95, 96, 169, 170, 173 artificial neural network (Abbr. ANN) An artificial network of nodes (or neurons) that can be tuned/trained from data to create valuable input-output mappings or categorization algo- rithms. For the purpose of this dissertation, this term is synonymous with a feedforward neural network with three layers (input, hidden, and output) that utilizes back propagation to tune the weights and biases of the sigmoidal activation functions of the nodes in the hidden layer. xix, xx, xxi, xxii, xxiii, xxiv, xxv, xxvi, 4, 6, 7, 8, 94, 95, 96, 99, 105, 106, 107, 108, 109, 110, 111, 112, 113, 116, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 141, 142, 143, 144, 145, 146, 147, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170 DOF Abbr. for degree of freedom. Usually referenced as the number of degrees of freedom, this refers to the dimensionality of the configuration space (e.g., joint angle space in tendon- driven systems). ix, xi, xix, 2, 12, 13, 29, 35, 100, 102, 110, 138 epoch In regards to the training of an artificial neural network, this refers to a single pass of all of the training data through the network. xxi, xxii, xxiv, 111, 131, 132, 137, 162, 163 joint stiffness The amount by which a rotational joint (like a pendulum) resists perturbations, calculated as the partial derivative of the joint torque with respect to the joint angle. xix, 172 xx, xxii, xxiii, xxv, 2, 104, 105, 106, 112, 113, 114, 115, 116, 117, 120, 133, 135, 136, 144, 146, 147, 150, 151, 152, 153, 154, 155, 156, 157, 161, 163, 164, 165, 166 kinematic redundancy The concept that serial linkage systems (like biological limbs or robotic arms) have more internal degrees of freedom (i.e., joint angles) than the number of degrees of interest at the endpoint (e.g., 3D position of 6D wrench), and therefore there are an infinite number of ways to orient the system to meet this requirement at the endpoint. This can be extended to entire movements (like initial value problems) where it can be said there are an infinite number of ways to complete a task in space (e.g., hitting the same spot with a hammer; Bernstein (1967)). 4, 7, 9, 10, 11, 28, 29, 30, 33, 50, 169 kinematic state Any state used to describe the limb dynamics (i.e., joint angles, angular veloci- ties, and angular accelerations). xxii, 3, 6, 7, 32, 95, 97, 136, 137, 141, 169, 173 mechanical parameter Those parameters that describe the system that governs the behavior of both sensory states (actuator states and tendon tensions) and kinematic states. These include, but are not limited to, joint inertia, tendon elasticity, motor damping. 7 motor babbling The process of learning a redundant input-output mapping by applying a limited amount of random inputs and recording the outputs. In biological systems, this is a proposed learning paradigm for infants when first learning to move. When discussing motor babbling as a means of training an artificial neural network to predict joint angle in a tendon-driven system, motor babbling refers to the low frequency, bandlimited white noise babbling de- scribed in Section 4.3.3. xix, xx, xxi, xxii, 4, 8, 95, 96, 99, 106, 107, 108, 120, 121, 122, 123, 127, 128, 129, 130, 131, 132, 133, 135, 137, 142, 145, 146, 150, 151, 154, 157, 167 muscle redundancy The concept that mammals have more muscles than they do controllable degrees of freedom (e.g., joints), requiring some form of dimensionality reduction like opti- mization to resolve this redundancy. 1, 4, 5, 10, 11, 27, 33, 60 musculotendon (Abbr. MT) The combination of both tendon and (pennated) muscle in series. ix, 173 xiv, xv, xxv, 6, 7, 9, 10, 11, 25, 32, 33, 35, 36, 37, 38, 39, 49, 50, 51, 53, 54, 56, 57, 58, 60, 61, 62, 63, 64, 65, 66, 70, 71, 73, 79, 80, 89, 90, 91, 92, 93, 138, 169, 195, 196, 198, 200, 201 musculotendon excursion The length change of the entire musculotendon complex as the direct result of joint rotations. In the assumption that musculotendon moment arms are functions of joint angles alone (i.e., not tension dependent), then musculotendon excursion is completely defined by the joint kinematics (See Section 3.3.3). ix, xii, xiv, xv, xvi, xvii, xviii, 6, 7, 9, 11, 18, 19, 20, 38, 53, 54, 55, 56, 59, 60, 63, 64, 65, 70, 71, 72, 73, 81, 82, 86, 88, 90, 92, 169, 196, 200 non-collocated Refers to sensors or sensory information that is not explicitly measured near the joint (e.g., motor angles are non-collocated measurements of pendulum angle in a rigid tendon-driven system). xxvi, 3, 4, 95, 96, 97, 99, 106, 109, 123, 132, 137, 138, 142, 166, 168 nullspace The subspace of the redundant variables that has no affect on the dynamics of a system. In the case of tendon-driven systems, the set of all tendon tensions that produce a net-zero torque on the limb(s). xiii, 32, 33, 34, 35, 48, 49, 61, 99, 106, 112 over-determined A term used to describe a system where there may be at most one viable solu- tions. Another way of visualizing this relationship is a mapping from few inputs to many outputs. 9, 10, 11, 19, 20, 25, 28, 29, 30, 31, 33 pennation angle Denoted in the text as, this parameter represents the angle between the line of action of the muscle fascicles and the musculotendon. Muscles that are more pennated will therefore exert large forces in a direction perpendicular to the tendon force. ix, xv, xvi, xvii, xviii, xxv, 6, 9, 10, 37, 39, 53, 62, 63, 65, 66, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 91, 92, 195, 196 relative degree The number of times we must differentiate the output before the input appears. If 174 this number is less than the order of the system dynamics then the system will have (poten- tially unstable) zero dynamics, which cannot be controlled by feedback linearization. See also zero dynamics, 116, 117, 175 sensory state A state relating to the actuator/tendon complex and not the joint itself (e.g., muscle lengths in biology or tendon tension in tendon-driven robots). Alternatively, a state that describes either tendon tension information or is an actuator state. 3, 7, 97, 113, 120, 127, 142, 167, 173 under-determined A term used to describe a system where there are infinitely many viable so- lutions (in different parts of the nullspace). Commonly used when referencing muscle re- dundancy where there are more muscles than joints and therefore an infinite combination of tendon forces that create the same net joint torques. Another way of visualizing this relationship is a mapping from many inputs to fewer outputs. 2, 4, 20, 32, 33 zero dynamics The dynamics of a system that are not controllable by feedback control. The dimensionality of the zero dynamics will be equal to the order of the system dynamics less the relative degree See also relative degree, 117, 175 175 References Adolph, K. E. (2008). Learning to move. Adolph, K. E., Cole, W. G., Komati, M., Garciaguirre, J. S., Badaly, D., Lingeman, J. M., Chan, G. L., and Sotsky, R. B. (2012). How Do You Learn to Walk? Thousands of Steps and Dozens of Falls per Day. Psychological Science. Agapaki, O. M., Anastasopoulos, D., Erimaki, S., and Christakos, C. N. (2016). Interaction of Stretch Reflex Loop with Descending B-Oscillations and the Generation of Tremor in Parkin- son‘s Disease: A Study of Motor Unit Firing Synchrony and Patterns. Neurology, 86(16 Supplement):5–369. Alexander, R. M. (1988). Elastic mechanisms in animal movement. Cambridge University Press, Cambridge. Alexander, R. M. and Bennet-Clark, H. C. (1977). Storage of elastic strain energy in muscle and other tissues. Nature, 265:114–117. Amis, A. A., Dowson, D., and Wright, V . (1979). Muscle strengths and musculo-skeletal geometry of the upper limb. Engineering in Medicine, 8(1):41–48. An, K., Ueba, Y ., Chao, E., Cooney, W., and Linscheid, R. (1983). Tendon excursion and moment arm of index finger muscles. Journal of Biomechanics, 16(6):419–425. An, K. N., Hui, F. C., Morrey, B. F., Linscheid, R. L., and Chao, E. Y . (1981). Muscles across the elbow joint: A biomechanical analysis. Journal of Biomechanics, 14(10):659–669. Andrychowicz, O. M., Baker, B., Chociej, M., J´ ozefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., Ray, A., Schneider, J., Sidor, S., Tobin, J., Welinder, P., Weng, L., and Zaremba, W. (2019). Learning dexterous in-hand manipulation. The International Journal of Robotics Research. 176 Appenteng, K. and Prochazka, A. (1984). Tendon organ firing during active muscle lengthening in awake, normally behaving cats. The Journal of Physiology, 353(1):81–92. Arampatzis, A., Karamanidis, K., Morey-Klapsing, G., De Monte, G., and Stafilidis, S. (2006). Mechanical properties of the triceps surae tendon and aponeurosis in relation to intensity of sport activity. Journal of Biomechanics, 40(9):1946–1952. Bennett, M. B., Ker, R. F., Imery, N. J., and Alexander, R. M. N. (1986). Mechanical properties of various mammalian tendons. Journal of Zoology, 209(4):537–548. Bernstein, N. A. (1967). The Coordination and Regulation of Movements. Pergamon Press, New York. Berret, B., Delis, I., Gaveau, J., and Jean, F. (2019). Optimality and modularity in human move- ment: From optimal control to muscle synergies. In Springer Tracts in Advanced Robotics, volume 124, pages 105–133. Springer Verlag. Berry, J. A., Ritter, R., Nagamori, A., and Valero-Cuevas, F. J. (2017). The neural control of movement must contend with trajectory-specific and nonlinearly distorted manifolds of afferent muscle spindle activity. In 2017 International Joint Conference on Neural Networks (IJCNN), pages 1188–1194. IEEE. Bicchi, A., Bavaro, M., Boccadamo, G., De Carli, D., Filippini, R., Grioli, G., Piccigallo, M., Rosi, A., Schiavi, R., Sen, S., and Tonietti, G. (2008). Physical human-robot interaction: De- pendability, safety, and performance. In International Workshop on Advanced Motion Control, AMC. Biewener, A. A. and Daley, M. A. (2007). Unsteady locomotion: Integrating muscle function with whole body dynamics and neuromuscular control. Blix, M. (1894). Die L¨ ange und die Spannung des Muskels. Skandinavisches Archiv F¨ ur Physi- ologie, 5(1):173–206. 177 Bongard, J., Zykov, V ., and Lipson, H. (2006). Resilient machines through continuous self- modeling. Science. Brand, P. W., Beach, R. B., and Thompson, D. E. (1981). Relative tension and potential excursion of muscles in the forearm and hand. The Journal of Hand Surgery, 6(3):2019–219. Brock, O. and Valero-Cuevas, F. (2016). Transferring synergies from neuroscience to robotics: Comment on Hand synergies: Integration of robotics and neuroscience for understanding the control of biological and artificial hands by M. Santello et al. Brown, I. E., Cheng, E. J., and Loeb, G. E. (1999). Measured and modeled properties of mam- malian skeletal muscle. II. The effects of stimulus frequency on force-length and force-velocity relationships. Journal of Muscle Research and Cell Motility, 20(7):627–643. Brown, I. E. and Loeb, G. E. (1999). Measured and modeled properties of mammalian skeletal muscle. I. The effects of post-activation potentiation on the time course and velocity dependen- cies of force production. Journal of Muscle Research and Cell Motility, 20(5-6):443–456. Brown, I. E., Scott, S. H., and Loeb, G. E. (1996). Mechanics of feline soleus: II Design and validation of a mathematical model. Journal of Muscle Research and Cell Motility, 17:221–233. Burdet, E., Osu, R., Franklin, D. W., Milner, T. E., and Kawato, M. (2001). The central nervous system stabilizes unstable dynamics by learning optimal impedance. Nature. Cheng, E. J., Brown, I. E., and Loeb, G. E. (2000). Virtual muscle: a computational approach to understanding the effects of muscle properties on motor control. Journal of Neuroscience Methods, 101:117–130. Crowe, A. and Matthews, P. B. (1964). The effects of stimulation of static and dynamic fusimotor fibres on the response to stretching of the primary endings of muscle spindles. The Journal of Physiology. 178 Crowninshield, R. D. and Brand, R. A. (1981). A physiologically based criterion of muscle force prediction in locomotion. Journal of Biomechanics. De Gooijer-Van De Groep, K. L., De Vlugt, E., Van Der Krogt, H. J., Helgad´ ottir, ., Arendzen, J. H., Meskers, C. G., and De Groot, J. H. (2016). Estimation of tissue stiffness, reflex activity, optimal muscle length and slack length in stroke patients using an electromyography driven antagonistic wrist model. Clinical Biomechanics, 35:93–101. de Rugy, A., Loeb, G. E., and Carroll, T. J. (2012). Muscle coordination is habitual rather than optimal. Journal of Neuroscience, 32(21):7384–7391. Di Carlo, J., Wensing, P. M., Katz, B., Bledt, G., and Kim, S. (2018). Dynamic Locomotion in the MIT Cheetah 3 Through Convex Model-Predictive Control. In IEEE International Conference on Intelligent Robots and Systems. Dimitriou, M. and Edin, B. B. (2008). Discharges in human muscle receptor afferents during block grasping. Journal of Neuroscience, 28(48):12632–12642. Dingwell, J. B., John, J., and Cusumano, J. P. (2010). Do humans optimally exploit redundancy to control step variability in walking? PLoS Computational Biology. Duchateau, J. and Enoka, R. M. (2016). Neural control of lengthening contractions. Journal of Experimental Biology, 219(2):197–204. Edgerton, V . R., Apor, P., and Roy, R. R. (1990). Specific tension of human elbow flexor muscles. Acta physiologica Hungarica, 75(3):205–16. Edin, B. B. (2001). Cutaneous afferents provide information about knee joint movements in hu- mans. Journal of Physiology, 531(1):289–297. Edin, B. B. and Abbs, J. H. (1991). Finger movement responses of cutaneous mechanoreceptors in the dorsal skin of the human hand. Journal of Neurophysiology, 65(3):657–670. 179 Edin, B. B. and Vallbo, A. B. (1990). Dynamic response of human muscle spindle afferents to stretch. Journal of Neurophysiology, 63(6):1297–1306. Fick, R. (1904). Handbuch der Anatomie und Mechanik der Gelenke unter Berucksichtigung der betwegenden Muskeln. Finni, T., Peltonen, J., Stenroth, L., and Cronin, N. J. (2013). Viewpoint: On the hysteresis in the human Achilles tendon. Journal of Applied Physiology, 114(4):515–517. Fitts, P. M. (1954). The information capacity of the human motor system in controlling the ampli- tude of movement. Journal of Experimental Psychology. Flash, T. and Hogan, N. (1985). The Coordination of Arm Movements: An Experimentally Con- firmed Mathematical Model’. The Journal of Neuroscience, 5(7):1688–1703. Friederich, J. A. and Brand, R. A. (1990). Muscle fiber architecture in the human lower limb. Journal of Biomechanics, 23(1):91–95. Fuglevand, A. J., Winter, D. A., and Patla, A. E. (1993). Models of recruitment and rate coding organization in motor-unit pools. Journal of Neurophysiology. Fukunaga, T., Ichinose, Y ., Ito, M., Kawakami, Y ., and Fukashiro, S. (1997). Determination of fascicle length and pennation in a contracting human muscle in vivo. Journal of Applied Physi- ology, 82(1):354–358. Gans, C. (1982). Fiber architecture and muscle function. Gans, C. and Bock, W. (1965). The functional significance of muscle architecture - a theoretical analysis. Advances in Anatomy, Embryology and Cell Biology, 38(April):115–142. Gareis, H., Solomonow, M., Baratta, R., Best, R., and D’Ambrosia, R. (1992). The isometric length-force models of nine different skeletal muscles. Journal of Biomechanics, 25:903–16. 180 Gijsberts, A. and Metta, G. (2013). Real-time model learning using Incremental Sparse Spectrum Gaussian Process Regression. Neural Networks. Gladwell, M. (2008). Outliers: The story of success. Hachette UK. Grieve, D. (1978). Prediction of gastrocnemius length from knee and ankle joint posture. Biome- chanics, A, 2:405–412. Hagen, D. A. and Valero-Cuevas, F. J. (2017). Similar movements are associated with drastically different muscle contraction velocities. Journal of Biomechanics, 59:90–100. Hahn, D., Olvermann, M., Richtberg, J., Seiberl, W., and Schwirtz, A. (2011). Knee and ankle joint torqueangle relationships of multi-joint leg extension. Journal of Biomechanics, 44(11):2059– 2065. Hatcher, D. D. and Luff, A. R. (1986). The effect of initial length on the shortening velocity of cat hind limb muscles. Pfl¨ ugers Archiv European Journal of Physiology, 407(4):396–403. Hatze, H. (1977). Biological Cybernetics A Myocybernetic Control Model of Skeletal Muscle. Technical report. Herbert, R. D. and Gandevia, S. C. (1995). Changes in pennation with joint angle and mus- cle torque: in vivo measurements in human brachialis muscle. The Journal of Physiology, 484(2):523–532. Herrick, W. C., Kingsbury, H. B., and Lou, D. Y . (1978). A study of the normal range of strain, strain rate, and stiffness of tendon. Journal of Biomedical Materials Research, 12(6):877–894. Herzog, W., Leonard, T. R., Renaud, J. M., Wallace, J., Chaki, G., and Bornemisza, S. (1992). Force-length properties and functional demands of cat gastrocnemius, soleus and plantaris mus- cles. Journal of Biomechanics, 25(11):1329–1335. Hidler, J. M. and Rymer, W. Z. (1999). A simulation study of reflex instability in spasticity: Origins of clonus. IEEE Transactions on Rehabilitation Engineering. 181 Hill, A. V . (1953). The mechanics of active muscle. Proceedings of the Royal Society of Lon- don. Series B, Containing papers of a Biological character. Royal Society (Great Britain), 141(902):104–117. Hof, A. L. (1998). In vivo measurement of the series elasticity release curve of human triceps surae muscle. Journal of Biomechanics, 31(9):793–800. Hof, A. L., Van Zandwijk, J. P., and Bobbert, M. F. (2002). Mechanics of human triceps surae muscle in walking, running and jumping. Acta Physiologica Scandinavica, 174(1):17–30. Hoffer, J. A., Caputi, A. A., Pose, I. E., and Griffiths, R. I. (1989). Roles of muscle activity and load on the relationship between muscle spindle length and whole muscle length in the freely walking cat. Progress in Brain Research. Holzbaur, K. R. S., Murray, W. M., and Delp, S. L. (2005). A Model of the Upper Extremity for Simulating Musculoskeletal Surgery and Analyzing Neuromuscular Control. Annals of Biomed- ical Engineering, 33(6):829–840. Hoy, M. G., Zajac, F. E., and Gordon, M. E. (1990). A Musculoskeletal Model of the Human Lower Extremity: The Effect of Muscle, Tendon, and Moment Arm on the Moment-Angle Relationship of Musculotendon Actuators at the Hip, Knee, and Ankle. Journal of Biomechanics, 23(2):157– 169. Huijing, P. A. (1985). Architecture of the human gastrocnemius muscle and some functional consequences. Cells Tissues Organs, 123(2):101–107. Hulliger, M., Nordh, E., Thelin, A. E., and Vallbo, A. B. (1979). The responses of afferent fibres from the glabrous skin of the hand during voluntary finger movements in man. The Journal of Physiology, 291(1):233–249. Hultborn, H. (2006). Spinal reflexes, mechanisms and concepts: From Eccles to Lundberg and beyond. 182 Hyun, D., Yang, H. S., Park, J., and Shim, Y . (2010). Variable stiffness mechanism for human- friendly robots. Mechanism and Machine Theory. Jacobsen, S., Iversen, E., Knutti, D., Johnson, R., and Biggers, K. (1986). Design of the Utah/M.I.T. Dextrous Hand. In Proceedings. 1986 IEEE International Conference on Robotics and Automation, pages 1520–1532. Jacobson, M. D., Raab, R., Fazeli, B. M., Abrams, R. A., Botte, M. J., and Lieber, R. L. (1992). Architectural design of the human intrinsic hand muscles. Journal of Hand Surgery, 17(5):804– 809. Jordan, M. I. and Wolpert, D. M. (1999). Computational motor control. The Cognitive Neuro- sciences. Joyce, G. C., Rack, P. M., and Westbury, D. R. (1969). The mechanical properties of cat soleus muscle during controlled lengthening and shortening movements. The Journal of Physiology, 204(2):461–474. Kandel, E. R. and Schwartz, J. H. (2000). Principles of Neural Science, volume 4. Kawakami, Y ., Ichinose, Y ., and Fukunaga, T. (1998). Architectural and functional features of human triceps surae muscles during contraction. Journal of Applied Physiology, 85:398–404. Ker, R. F. (1981). Dynamic tensile properties of the plantaris tendon of sheep (Ovis aries). The Journal of experimental biology, 93:283–302. Kistemaker, D. A., Knoek van Soest, A. J., Wong, J. D., Kurtzer, I., and Gribble, P. L. (2013). Control of position and movement is simplified by combined muscle spindle and Golgi tendon organ feedback. Journal of Neurophysiology, 109(4):1126–1139. Knudson, D. (1993). Biomechanics of the Basketball Jump ShotSix Key Teaching Points. Journal of Physical Education, Recreation & Dance, 64(2):67–73. 183 Ko, Y . G., Challis, J. H., and Newell, K. M. (2003). Learning to coordinate redundant degrees of freedom in a dynamic balance task. Human Movement Science. Kobayashi, H., Hyodo, K., and Ogane, D. (1998). On tendon-driven robotic mechanisms with redundant tendons. International Journal of Robotics Research, 17(5):561–571. Konow, N., Azizi, E., and Roberts, T. J. (2012). Muscle power attenuation by tendon during energy dissipation. Proceedings of the Royal Society B: Biological Sciences, 279(1731):1108–1113. Kurokawa, S., Fukunaga, T., and Fukashiro, S. (2001). Behavior of fascicles and tendinous structures of human gastrocnemius during vertical jumping. Journal of Applied Physiology, 90(4):1349–1358. Kurse, M. U., Lipson, H., and Valero-Cuevas, F. J. (2012). Extrapolatable analytical functions for tendon excursions and moment arms from sparse datasets. IEEE Transactions on Biomedical Engineering. Kutch, J. J. and Valero-Cuevas, F. J. (2012). Challenges and new approaches to proving the exis- tence of muscle synergies of neural origin. PLoS Computational Biology. Kwah, L. K., Pinto, R. Z., Diong, J., and Herbert, R. D. (2013). Reliability and validity of ultra- sound measurements of muscle fascicle length and pennation in humans: A systematic review. Kwiatkowski, R. and Lipson, H. (2019). Task-agnostic self-modeling machines. Science Robotics, 4(26):eaau9354. Laine, C. M., Nagamori, A., and Valero-Cuevas, F. J. (2016). The dynamics of voluntary force production in afferented muscle influence involuntary tremor. Frontiers in Computational Neu- roscience. Laurin-Kovitz, K. F., Colgate, J. E., and Carnes, S. D. R. (1991). Design of components for programmable passive impedance. In Proceedings - IEEE International Conference on Robotics and Automation, volume 2, pages 1476–1481. 184 Lee, J. J. and Tsai, L. W. (1991). Structural synthesis of tendon-driven manipulators having a pseudotriangular structure matrix. International Journal of Robotics Research, 10(3):255–262. Li, W., Todorov, E., and Pan, X. (2004). Hierarchical optimal control of redundant biomechanical systems. In Annual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings. Lichtwark, G. A. and Wilson, A. (2005). In vivo mechanical properties of the human Achilles tendon during one-legged hopping. Journal of Experimental Biology, 208(24):4715–4725. Lichtwark, G. A. and Wilson, A. M. (2007). Is Achilles tendon compliance optimised for maximum muscle efficiency during locomotion? Journal of Biomechanics, 40:1768–1775. Lichtwark, G. A. and Wilson, A. M. (2008). Optimal muscle fascicle length and tendon stiffness for maximising gastrocnemius efficiency during human walking and running. Journal of Theoretical Biology, 252(4):662–673. Lieber, R. L. and Brown, C. G. (1992). Quantitative method for comparison of skeletal muscle architectural properties. Journal of Biomechanics, 25(5):557–560. Lieber, R. L., Fazeli, B. M., and Botte, M. J. (1990). Architecture of selected wrist flexor and extensor muscles. Journal of Hand Surgery. Loeb, G. E. (1984). The Control and Responses of Mammalian Muscle Spindles During Normally Executed Motor Tasks. Exercise and Sport Sciences Reviews, 12(1):157–204. Loeb, G. E. (2000). Overcomplete musculature or underspecified tasks? Loeb, G. E. (2012). Optimal isnt good enough. Biological Cybernetics, 106(11-12):757–765. Loeb, G. E., Levine, W. S., and He, J. (1990). Understanding sensorimotor feedback through optimal control. In Cold Spring Harbor Symposia on Quantitative Biology. 185 Lohse, K. R., Lang, C. E., and Boyd, L. A. (2014). Is more better? Using metadata to explore dose-response relationships in stroke rehabilitation. Stroke. Maganaris, C. N. and Baltzopoulos, V . (1999). Predictability of in vivo changes in pennation angle of human tibialis anterior muscle from rest to maximum isometric dorsiflexion. European Journal of Applied Physiology and Occupational Physiology, 79(3):294–297. Maganaris, C. N., Baltzopoulos, V ., and Sargeant, A. J. (1998). In vivo measurements of the triceps surae complex architecture in man: Implications for muscle function. Journal of Physiology. Maganaris, C. N. and Paul, J. P. (2000). Hysteresis measurements in intact human tendon. Journal of Biomechanics, 33(12):1723–1727. Maganaris, C. N. and Paul, J. P. (2002). Tensile properties of the in vivo human gastrocnemius tendon. Journal of Biomechanics, 35(12):1639–1646. Magnusson, S. P., Aagaard, P., Rosager, S., Dyhre-Poulsen, P., and Kjaer, M. (2001). Load- displacement properties of the human triceps surae aponeurosis ¡i¿in vivo¡/i¿. The Journal of Physiology, 531(1):277–288. Mah, C. D. and Mussa-Ivaldi, F. A. (2003). Evidence for a specific internal representation of motion-force relationships during object manipulation. Biological Cybernetics. Marjaninejad, A., Annigeri, R., and Valero-Cuevas, F. J. (2018). Model-Free Control of Movement in a Tendon-Driven Limb via a Modified Genetic Algorithm. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, volume 2018-July, pages 1767–1770. Marjaninejad, A., Tan, J., and Valero-Cuevas, F. (2019a). Autonomous Control of a Tendon- driven Robotic Limb with Elastic Elements Reveals that Added Elasticity can Enhance Learning. Technical report, University of Southern California. 186 Marjaninejad, A., Urbina-Mel´ endez, D., Cohn, B. A., and Valero-Cuevas, F. J. (2019b). Au- tonomous functional movements in a tendon-driven limb via limited experience. Nature Ma- chine Intelligence, 1(3):144–154. Marjaninejad, A., Urbina-Melfn’feggndez, D., and Valero-Cuevas, F. J. (2019c). Simple Kine- matic Feedback Enhances Autonomous Learning in Bio-Inspired Tendon-Driven Systems. arXiv preprint arXiv:1907.04539. Marjaninejad, A. and Valero-Cuevas, F. J. (2019). Should anthropomorphic systems be Redundant? In Springer Tracts in Advanced Robotics, volume 124, pages 7–34. Martin, D. C., Medri, M. K., Chow, R. S., Oxorn, V ., Leekam, R. N., Agur, A. M., and McKee, N. H. (2001). Comparing human skeletal muscle architectural parameters of cadavers with in vivo ultrasonographic measurements. Journal of Anatomy, 199(4):429–434. Mazumdar, A., Spencer, S. J., Hobart, C., Salton, J., Quigley, M., Wu, T., Bertrand, S., Pratt, J., and Buerger, S. P. (2017). Parallel Elastic Elements Improve Energy Efficiency on the STEPPR Bipedal Walking Robot. IEEE/ASME Transactions on Mechatronics. Menegaldo, L. L., de Toledo Fleury, A., and Weber, H. I. (2004). Moment arms and musculo- tendon lengths estimation for a three-dimensional lower-limb model. Journal of Biomechanics, 37(9):1447–1453. Mileusnic, M. P. and Loeb, G. E. (2006). Mathematical models of proprioceptors. II. Structure and function of the Golgi tendon organ. Journal of Neurophysiology, 96(4):1789–1802. Millard, M., Uchida, T., Seth, A., and Delp, S. L. (2013). Flexing Computational Muscle: Modeling and Simulation of Musculotendon Dynamics. Journal of Biomedical Engineering, 135(2):021005–021005. Milner, T. E. (2002). Contribution of geometry and joint stiffness to mechanical stability of the human arm. Experimental Brain Research. 187 Muraoka, T., Muramatsu, T., Fukanaga, T., and Kanehisa, H. (2005). Elastic properties of human Achilles tendon are correlated to muscle strength. Journal of Applied Physiology, 99:665–669. Mussa-Ivaldi, F. A., Hogan, N., and Bizzi, E. (1985). Neural, mechanical, and geometric factors subserving arm posture in humans. Journal of Neuroscience. Narici, M. V ., Binzoni, T., Hiltbrand, E., Fasel, J., Terrier, F., and Cerretelli, P. (1996a). In vivo human gastrocnemius architecture with changing joint angle at rest and during graded isometric contraction. Journal of Physiology. Narici, M. V ., Binzoni, T., Hiltbrand, E., Fasel, J., Tettier, F., and Cerretelli, P. (1996b). Changes in human gastrocnemius architecture with joint angle at rest and with isometric contraction, evaluated in vivo. Journal of Physiology (London), 496(1):287–297. Newell, K. M. and Vaillancourt, D. E. (2001). Dimensional change in motor learning. Human Movement Science. Nicolescu, M. N. and Matari´ c, M. J. (2003). Natural Methods for Robot Task Learning: Instructive Demonstrations, Generalization and Practice. In Proceedings of the International Conference on Autonomous Agents. Osu, R. and Gomi, H. (1999). Multijoint muscle regulation mechanisms examined by measured human arm stiffness and EMG signals. Journal of Neurophysiology. Otten, E. (1988). Concepts and models of functional architecture in skeletal muscle. Exercise and Sport Sciences Reviews, 16(1):89–137. Pain, M. T., Young, F., Kim, J., and Forrester, S. E. (2013). The torquevelocity relationship in large human muscles: Maximum voluntary versus electrically stimulated behaviour. Journal of Biomechanics, 46(4):645–650. Palli, G., Borghesan, G., and Melchiorri, C. (2012). Modeling, identification, and control of tendon-based actuation systems. IEEE Transactions on Robotics, 28(2):277–290. 188 Palli, G., Melchiorri, C., and De Luca, A. (2008). On the feedback linearization of robots with variable joint stiffness. In Proceedings - IEEE International Conference on Robotics and Au- tomation, pages 1753–1759. Palli, G., Melchiorri, C., Wimb¨ ock, T., Grebenstein, M., and Hirzinger, G. (2007). Feedback linearization and simultaneous stiffness-position control of robots with antagonistic actuated joints. In Proceedings - IEEE International Conference on Robotics and Automation, number April, pages 4367–4372. Perreault, E. J., Heckman, C. J., and Sandercock, T. G. (2003). Hill muscle model errors dur- ing movement are greatest within the physiologically relevant range of motor unit firing rates. Journal of Biomechanics. Perreault, E. J., Kirsch, R. F., and Crago, P. E. (2001). Effects of voluntary force generation on the elastic components of endpoint stiffness. Experimental Brain Research. Perreault, E. J., Kirsch, R. F., and Crago, P. E. (2002). V oluntary control of static endpoint stiffness during force regulation tasks. Journal of Neurophysiology. Phadke, C. P., Flynn, S., Kukulka, C., Thompson, F. J., and Behrman, A. L. (2016). Comparison of soleus H-Reflexes in two groups of individuals with motor incomplete spinal cord injury walking with and without a walker. Topics in Spinal Cord Injury Rehabilitation. Pollock, C. M. and Shadwick, R. E. (2017). Relationship between body mass and biomechanical properties of limb tendons in adult mammals. American Journal of Physiology-Regulatory, Integrative and Comparative Physiology. Pratt, G. A. (2002). Low impedance walking robots. In Integrative and Comparative Biology. Pratt, G. A. and Williamson, M. M. (1995). Series elastic actuators. IEEE International Conference on Intelligent Robots and Systems, 1:399–406. Prilutsky, B. I. (2000). Muscle Coordination: The Discussion Continues. Motor Control. 189 Prochazka, A., Hulliger, M., Zangger, P., and Appenteng, K. (1985). Fusimotor set: new evi- dence for-independent control of -motoneurones during movement in the awake cat. Brain Research, 339(1):136–140. Proske, U. and Morgan, D. (1987). Tendon stiffness: Methods of measurement and significance for the control of movement. A review. Journal of Biomechanics, 20(1):75–82. Rack, P. M. and Westbury, D. R. (1969). The effects of length and stimulus rate on tension in the isometric cat soleus muscle. The Journal of Physiology, 204(2):443–460. R´ acz, K. and Valero-Cuevas, F. J. (2013). Spatio-temporal analysis reveals active control of both task-relevant and task-irrelevant variables. Frontiers in Computational Neuroscience. Ramsay, J. W., Hunter, B. V ., and Gonzalez, R. V . (2009). Muscle moment arm and normal- ized moment contributions as reference data for musculoskeletal elbow and wrist joint models. Journal of Biomechanics, 42(4):463–473. Rankin, J. W. and Neptune, R. R. (2012). Musculotendon lengths and moment arms for a three- dimensional upper-extremity model. Journal of Biomechanics, 45(9):1739–1744. Roberts, T. J. and Azizi, E. (2010). The series-elastic shock absorber: tendons attenuate muscle power during eccentric actions. Journal of Applied Physiology, 109(2):396–404. Roy, R. R. and Edgerton, V . R. (1992). Skeletal Muscle architecture and performance. Strength and Power in Sport, pages 115–129. Sanger, T. D., Chen, D., Fehlings, D. L., Hallett, M., Lang, A. E., Mink, J. W., Singer, H. S., Alter, K., Ben-Pazi, H., Butler, E. E., Chen, R., Collins, A., Dayanidhi, S., Forssberg, H., Fowler, E., Gilbert, D. L., Gorman, S. L., Gormley, M. E., Jinnah, H. A., Kornblau, B., Krosschell, K. J., Lehman, R. K., MacKinnon, C., Malanga, C. J., Mesterman, R., Michaels, M. B., Pearson, T. S., Rose, J., Russman, B. S., Sternad, D., Swoboda, K. J., and Valero-Cuevas, F. (2010). Definition and classification of hyperkinetic movements in childhood. 190 Santos, V . J., Bustamante, C. D., and Valero-Cuevas, F. J. (2009). Improving the fitness of high- dimensional biomechanical models via data-driven stochastic exploration. IEEE Transactions on Biomedical Engineering. Scholz, J. P. and Sch¨ oner, G. (1999). The uncontrolled manifold concept: Identifying control variables for a functional task. Experimental Brain Research. Scott, S. H. (2004). Optimal feedback control and the neural basis of volitional motor control. Scott, S. H., Engstrom, C. M., and Loeb, G. E. (1993). Morphometry of human thigh muscles. Determination of fascicle architecture by magnetic resonance imaging. Journal of anatomy, 182 ( Pt 2:249–57. Scott, S. H. and Loeb, G. E. (1994). The computation of position sense from spindles in mono- and multiarticular muscles. Journal of Neuroscience, 14(12):7529–7540. Scott, S. H. and Loeb, G. E. (1995). Mechanical properties of aponeurosis and tendon of the cat soleus muscle during wholemuscle isometric contractions. Journal of Morphology, 224(1):73– 86. Scovil, C. Y . and Ronsky, J. L. (2006). Sensitivity of a Hill-based muscle model to perturbations in model parameters. Journal of Biomechanics. Shadwick, R. E. (1990). Elastic energy storage in tendons: Mechanical differences related to function and age. Journal of Applied Physiology. Sherrington, C. S. (1913). Reflex inhibition as a factor in the co-ordination of movements and postures. Quarterly Journal of Experimental Physiology, 6(3):251–310. Sherrington, C. S. (1932). Inhibition as a coordinative factor. N. Foundation (Ed.), Nobel Lectures, Physiology and Medicine, pages 278–289. 191 Song, D., Lan, N., Loeb, G. E., and Gordon, J. (2008a). Model-based sensorimotor integration for multi-joint control: Development of a virtual arm model. Annals of Biomedical Engineering, 36(6):1033–1048. Song, D., Raphael, G., Lan, N., and Loeb, G. E. (2008b). Computationally efficient models of neuromuscular recruitment and mechanics. Journal of Neural Engineering, 5(5):175–184. Spelke, E. S. and Kinzler, K. D. (2007). Core knowledge. Spinka, M., Newberry, R. C., and Bekoff, M. (2001). Mammalian play: training for the unexpected. The Quarterly review of biology, 76(2):141–68. Stanev, D. and Moustakas, K. (2019). Modeling musculoskeletal kinematic and dynamic redun- dancy using null space projection. PLoS ONE, 14(1):1–26. Stevens, H. C. and Snodgrass, J. M. (1932). The Force-Velocity Curve of Striated Muscle. Pro- ceedings of the Society for Experimental Biology and Medicine, 30(3):373–375. Theodorou, E. and Valero-Cuevas, F. J. (2010). Optimality in neuromuscular systems. In 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC’10, pages 4510–4516. Ting, L. H. (2007). Dimensional reduction in sensorimotor systems: a framework for understand- ing muscle coordination of posture. Todorov, E. and Jordan, M. I. (2002). Optimal feedback control as a theory of motor coordination. Nature Neuroscience. Tonietti, G., Schiavi, R., and Bicchi, A. (2005). Design and Control of a Variable Stiffness Actua- tor. Proceedings of the 2005 IEEE ICRA International Conference on Robotics and Automation, (April):1–6. Valero-Cuevas, F. J. (2016). Fundamentals of Neuromechanics, volume 8 of Biosystems & Biorobotics. Springer London, London. 192 Valero-Cuevas, F. J., Cohn, B. A., Yngvason, H. F., and Lawrence, E. L. (2015). Exploring the high-dimensional structure of muscle redundancy via subject-specific and generic musculoskele- tal models. Journal of Biomechanics, 48:2887–2896. Valero-Cuevas, F. J., Hoffmann, H., Kurse, M. U., Kutch, J. J., and Theodorou, E. A. (2009). Computational Models for Neuromuscular Function. IEEE Reviews in Biomedical Engineering. Van Soest, A. J. and Rozendaal, L. A. (2008). The inverted pendulum model of bipedal standing cannot be stabilized through direct feedback of force and contractile element length and velocity at realistic series elastic element stiffness. Biological Cybernetics, 99(1):29–41. Venkadesan, M. and Valero-Cuevas, F. J. (2008). Neural Control of Motion-to-Force Transitions with the Fingertip. The Journal of Neuroscience, 28(6):1366–1373. Ward, S. R., Eng, C. M., Smallwood, L. H., and Lieber, R. L. (2009). Are current measurements of lower extremity muscle architecture accurate? Clinical Orthopaedics and Related Research, 467(4):1074–1082. Wickiewicz, T. L., Roy, R. R., Powell, P. L., and Edgerton, V . R. (1983). Muscle Architecture of the Human Lower Limb. Clinical Orthopaedics and Related Research, 179(1):275???283. Winter, D. A. (2009). Biomechanics and motor control of human movement. Wiley. Yamaguchi, G., Sawa, A., Moran, D., Fessler, M., and Winters, J. (1990). A Survey of Human Musculotendon Actuator Paramters. Multiple Muscle Systems: Biomechanics and Movement Organization, pages 717–773. Yoshikawa, T. (1990). Foundations of robotics: analysis and control. MIT Press. Zahalak, G. I. (1990). Modeling Muscle Mechanics (and Energetics). In Multiple Muscle Systems, pages 1–23. Zajac, F. E. (1989). Muscle and tendon: properties, models, scaling, and application to biome- chanics and motor control. Critical Reviews in Biomedical Engineering, 17(4):359–410. 193 Zehr, E. P. and Stein, R. B. (1999). What functions do reflexes serve during human locomotion? Zinn, M., Khatib, O., Roth, B., and Salisbury, J. K. (2004). Playing it safe. IEEE Robotics and Automation Magazine, 11(2):12–21. 194 Appendices A Integrator Backstepping Example Source Code The code used in this study and additional figures can be accessed through the project’s Github repository at: https://github.com/danhagen/NonlinearControl. B Addressing the Limitations of the “Parallelogram” Muscle Assumption In Chapter 3, a reasonable assumption about the architecture of the MT was made in order to better understand the relationship between muscle fascicle, tendon, and MT. Mainly, that the shape of the MT (or more specifically, the muscle “belly”) resembled a parallelogram with a constant height to ensure constant muscle “volume” (here approximated as area) and to ensure that pennated l m l MT = l T,1 + l T,2 + l T,int cos( β) + l m cos(ρ) = l T,ext + L mb l T,1 l T,2 d (const.) ρ β l T,int (const.) l T,int (const.) L mb Figure B.1: Approximation of musculotendon (MT) geometry as a flattened parallel bundle of pennated muscle fascicles in series with tendon such that the (constant) internal tendon length is not necessarily parallel to the external tendon (askew by angle). The muscle fascicles are still askew from the line of action of the MT by the pennation angle, . The projected length of the entire muscle “belly” is defined asL mb and is equal to the internal tendon and the muscle fascicle length (l m ) being projected back onto the line of action (Gans and Bock, 1965; Gans, 1982; Otten, 1988; Zajac, 1989). 195 muscle fascicles produce the same work as the entire muscle (Fick, 1904). This assumption was made for the sake of computational simplicity and because it allowed for a simple comparison to be made with models that typically use this approximation, but it neglects the fact that muscles will twist and bulge during contractions and that the aponeurosis or internal segment of the tendon is not always parallel to the external tendon (Figure B.1). From this relationship, it is easy to see that the change in MT (i.e. , its excursion) can be calculated as the sum of the changes in external tendon length (l T;ext ) and the muscle “belly” length (L mb ). Note that in Chapter 3, we lump the internal segment of the tendon in with the external segment (as they are parallel) and the entire tendon length was denoted asl T . This inclusion is also supported by the claims that the aponeurosis has a similar stiffness to the external tendon stiffness and the entire tendon, therefore, stretches relatively consistently across its entire length(Scott and Loeb, 1995). However, in the work documented in Otten (1988), this internal tendon length (l T;int ) is considered constant, which allowed them to calculate an explicit relationship between muscle fascicle lengths and the muscle belly length. The derivation from Otten (1988) is reproduced below and extended to discuss the change in muscle “belly” length. First we consider that the work of the muscle fascicles (W m ) is equal to the work of the entire muscle (W mb ). W m =W mb f m l m =f mb L mb f m l m =f T L mb (B.1a) (B.1b) (B.1c) However, the force of muscle belly is equal to the tendon tension and equal to the muscle fascicle force projected back onto the line of action of the MT by the cosine of the pennation angle. f T =f m cos() (B.2) 196 We therefore rewrite Eq. B.1 as, f m l m =f m cos()L mb l m = cos()L mb (B.3a) (B.3b) From Figure B.1, we can redefined the cosine of by the rule of cosine (Eq. B.4). cos() = l 2 m +L 2 mb l 2 T;int 2l m L mb (B.4) Therefore, Otten arrived at the following equation relating the muscle fascicles to the muscle belly. l m (2l m L mb ) = L mb (l 2 m +L 2 mb l 2 T;int ) (B.5) They rewrote this equality in the infinitesimal form (Eq. B.6) and solved it to produce the relation- ship given in Eq. B.7. @l m (2l m L mb ) =@L mb (l 2 m +L 2 mb l 2 T;int ) (B.6) L mb =c + q c 2 +l 2 m l 2 T;int (B.7) where c = l 2 T;int l 2 m;o +L 2 mb;o 2L mb;o where the subscript “o” indicates measurements taken at the muscles optimal length. This relationship can be further expanded from what was presented in Otten (1988) to present a more intuitive interpretation of this equation. First, we rewritec as Eq. B.8 to show that it is the 197 internal segment of the tendon projected back onto the line of action of the MT by the cosine of the angle, o . c = l 2 T;int l 2 m;o +L 2 mb;o 2L mb;o = l 2 m;o l 2 T;int L 2 mb;o 2L mb;o = l 2 m;o +L 2 mb;o l 2 T;int 2L mb;o L mb;o =L mb;o l m;o cos( o ) =l T;int cos( o ) (B.8a) (B.8b) (B.8c) (B.8d) We can similarly manipulate Eq. B.7 to find a more intuitive relationship between the muscle belly length and the muscle fascicle lengths (Eq. B.9). L mb =c + q c 2 +l 2 m l 2 T;int =c + q c 2 +l 2 m l 2 T;int =c + s c 2 + 2l m L mb l 2 m +L 2 mb l 2 T;int 2l m L mb L 2 mb =c + q c 2 + 2l m L mb cos()L 2 mb =c + q c 2 l 2 m cos 2 () 2l m L mb cos() +L 2 mb +l 2 m cos 2 () =c + q c 2 L mb l m cos() 2 +l 2 m cos 2 () =c + q l 2 T;int cos 2 ( o )l 2 T;int cos 2 () +l 2 m cos 2 () =c + q l 2 T;int cos 2 ( o ) cos 2 () +l 2 m cos 2 () =L mb;o l m;o cos( o ) + q l 2 T;int sin( o +) sin( o ) +l 2 m cos 2 () (B.9a) (B.9b) (B.9c) (B.9d) (B.9e) (B.9f) (B.9g) (B.9h) (B.9i) 198 Therefore, we can rewrite this equation to generalize to the change in muscle belly length; L mb = q l 2 T;int sin( o +) sin( o ) +l 2 m cos 2 ()l m;o cos( o ) =l m cos()l m;o cos( o ) (when = o is constant) (B.10a) (B.10b) Note that the approximation used in Chapter 3 is recovered when (i) the angle by which the aponeu- rosis is askew,, is zero everywhere (the original assumption) as well as when (ii) is constant. This relationship can be seen in Figure B.2. 0 30 60 90 -0.75 -0.50 -0.25 0.00 0.25 0.50 0.75 Internal Tendon Deflection Angle ( β in deg.) Initial Internal Tendon Deflection Angle ( β o in deg.) 0 30 Coefficient to l T,int in Eq. B.15a (sin( β o + β) sin( β o - β)) 2 90 90 Figure B.2: Contour plot depicting the value of the coefficient of l 2 T;int in Eq. B.10a (sin( o + ) sin( o )), which captures the vector contribution of the internal segment of the tendon as the angle by which it is askew () changes with respect to muscle contraction. Note that this value is near zero when the angles do not change by very much. 199 l MT l T = L mb = q l 2 T;int sin( o +) sin( o ) +l 2 m cos 2 ()l m;o cos( o ) ) l m = 1 cos() q l MT l T +l m;o cos( o ) 2 l 2 T;int sin( o +) sin( o ) (B.11a) (B.11b) (B.11c) Therefore the real assumptions made in Chapter 3 with respect to the MT architecture were that (i) the internal tendon did not change length (to satisfy the constant volume condition and the condition that muscle work is equal to the muscle fascicle work) and (ii) the angle by which the internal tendon is askew from the line of action of the MT (or the amount by which it changes with respect to muscle contraction) was negligible. If we assume that the belly of the muscle must shorten by some amount L mb (given by the MT excursion minus the external tendon deforma- tion), then if > o (i.e., the muscle bulges with a concentric contraction and the internal tendon deflects more from the MT line of action) the value of sin( o +) sin( o ) is negative and the equation derived in Chapter 3 would estimate the muscle fascicle length to be shorter than it really is (i.e., overestimate the amount of shortening). Similarly, if the muscle was to lengthen by some value of L mb , then if < o (i.e., the muscle narrows with an eccentric contraction) the value of sin( o +) sin( o ) is positive and the equation derived in Chapter 3 would estimate the muscle fascicle length to be longer than it really is (i.e., overestimating the amount of lengthening). This overestimation of the change in muscle fascicle length could create an overestimation in muscle activations if the actual muscle fascicle lengths are closer to the plateau of the muscle fascicle’s force-length relationship. This analysis extends loosely to the modelling of muscle fibers that are allowed to bow with contraction to satisfy the minimal changes in volume seen in Nature. Mainly, it stands to reason that a model that lumps all muscle fascicle fibers into one that does not consider the amount by which the orientation of the fibers relative to the internal tendon change with contraction will 200 likely exaggerate the behavior of the fascicles as the shortest path is the one between the two plates of tendinous tissue. Therefore, we find that the “better” muscle fascicle length equation derived in Chapter 3 is just that, “better”. It has limitations in the assumptions that were made which prevent it from completely describing the behavior of muscle fascicles during contraction but it does provide a better approximation when using the common assumption that tendons are parallel to the line of action of the MT. C insideOut Algorithm Source Code The code used in this study (including the feedback linearization algorithm) and some addi- tional figures can be accessed through the project’s Github repository at: https://github.com/danhagen/insideOut. 201
Abstract (if available)
Abstract
Estimates of limb posture are critical for the control of robotic systems. This is generally accomplished by utilizing on-location joint angle encoders which may complicate the design, increase limb inertia, and add noise to the system. Conversely, some innovative or smaller robotic morphologies can benefit from non-collocated sensors when encoder size becomes prohibitively large or the joints are less accessible or subject to damage (e.g., distal joints of a robotic hand or foot sensors subject to repeated impact). These concerns are especially important for tendon-driven systems where motors (and their sensors) are not placed at the joints. Here we create a novel framework that can utilize limited-experience to provide accurate and efficient joint angle estimation during dynamic tasks using non-collocated actuator and tendon tension measurements. We draw inspiration from Nature where (i) muscles and tendons have mechanoreceptors, (ii) there are no dedicated joint-angle sensors, and (iii) dedicated neural networks may perform sensory fusion. ❧ To do so, we simulated an inverted pendulum driven by an agonist-antagonist pair of motors that pull on tendons with nonlinear elasticity, then compared the contributions of different sets of non-collocated sensory information (like motor positions or tendon tensions) when training artificial neural networks (ANNs) to estimate joint angle. By comparing performance across different movement tasks we were able to determine how well each ANNs (trained on the different sensory sets of babbling data) generalizes to unlearned tasks (sinusoidal and point-to-point). We find that training an ANN with motor positions and velocities as well as tendon tension data produces more accurate estimates of joint angles than those ANNs trained without tendon tension data across all generalization movements. Additionally, we find that these results are robust to changes in mechanical plant parameters like tendon stiffness and motor damping and performs consistently well across different types of movements (from slow to ballistic). We conclude that regardless of the tendon architecture, the actuator behavior, or the movement complexity, tendon tension information is useful (if not vital) when estimating joint angles from non-collocated sensory signals. Additionally, this work—which shows that tendon tension information is invaluable for the estimation of joint angles by a simple neural network for a tendon-driven system—provides evidence to support the notion that in biological systems tendon tension information (conveyed by Golgi tendon organs) can be used with spindle afferents to better estimate limb posture.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Bio-inspired tendon-driven systems: computational analysis, optimization, and hardware implementation
PDF
Exploiting mechanical properties of bipedal robots for proprioception and learning of walking
PDF
A non-invasive and intuitive command source for upper limb prostheses
PDF
Design and use of a biomimetic tactile microvibration sensor with human-like sensitivity and its application in texture discrimination using Bayesian exploration
PDF
Inference of computational models of tendon networks via sparse experimentation
PDF
Development and control of biologically-inspired robots driven by artificial muscles
PDF
Geometric and dynamical modeling of multiscale neural population activity
PDF
Neuromuscular dynamics in the context of motor redundancy
PDF
High throughput computational framework for synthesis and accelerated discovery of dielectric polymer materials using polarizable reactive molecular dynamics and graph neural networks
Asset Metadata
Creator
Hagen, Daniel August
(author)
Core Title
insideOut: Estimating joint angles in tendon-driven robots using Artificial Neural Networks and non-collocated sensors
School
Viterbi School of Engineering
Degree
Doctor of Philosophy
Degree Program
Biomedical Engineering
Publication Date
09/02/2020
Defense Date
06/19/2020
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
Ann,artificial neural networks,joint kinematics,machine learning,muscle mechanics,musculotendon dynamics,OAI-PMH Harvest,tendon-driven robots
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Valero-Cuevas, Francisco J. (
committee chair
), Jovanovic, Mihailo R. (
committee member
), Loeb, Gerald E. (
committee member
)
Creator Email
daniel.8.hagen@gmail.com,dhagen@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c89-366470
Unique identifier
UC11666461
Identifier
etd-HagenDanie-8935.pdf (filename),usctheses-c89-366470 (legacy record id)
Legacy Identifier
etd-HagenDanie-8935.pdf
Dmrecord
366470
Document Type
Dissertation
Rights
Hagen, Daniel August
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
artificial neural networks
joint kinematics
machine learning
muscle mechanics
musculotendon dynamics
tendon-driven robots