Page 247 |
Save page Remove page | Previous | 247 of 289 | Next |
|
small (250x250 max)
medium (500x500 max)
Large (1000x1000 max)
Extra Large
large ( > 500x500)
Full Resolution
All (PDF)
|
This page
All
|
were used to describe the kinematics of the hand trajectory (Harris & Wolpert 1998),
(Todorov & Jordan 2002). In neuromuscular modeling, however, linear models cannot
capture the nonlinear behavior of muscles and multi- body limbs. In (Li & Todorov 2004),
an Iterative Linear Quadratic Regulator (ILQR) was first introduced for the optimal con-trol
of nonlin- ear neuromuscular models. The proposed method is based on linearization
of the dynamics. An interesting component of this work that played an influential role in
the studies on optimal control methods for neuromuscular models was the fact that there
was no need for a pre-specified desired trajectory in state space.
By contrast, most approaches for neuromuscular optimization that use classical con-trol
theory (see Section VI) require target time histories of limb kinematics, kinetics
and/or muscle activity. In (Todorov 2005) the ILQR method was extended for the case
of nonlinear stochastic systems with state and control dependent noise. The proposed
algorithm is the Iterative Linear Quadratic Gaussian Regulator (iLQG). This extension
allows the use of stochastic nonlinear models for muscle force as a function of fiber length
and fiber velocity. Figure 6 illustrates the application of LQG to our arm model (Section
II). Further theoretical developments in (Li & Todorov 2006) and (Todorov 2007) allowed
the use of an Extended Kalman Filter (EKF) for the case of sensory feedback noise. The
EKF is an extension of the Kalman filter for nonlinear systems.
The has been only few examples of studies in the area of the biomechanics of the index
finger which try to identity the underlying control signals for the case of movement and
force production, either these signals corresponds to neural commands or tensions ap-plied
on the tendons. More precisely on the experimental side, the work in (Venkadesan
& Valero-Cuevas 2008b) investigated the neural control of contact transition between
233
Object Description
| Title | Iterative path integral stochastic optimal control: theory and applications to motor control |
| Author | Theodorou, Evangelos A. |
| Author email | etheodor@usc.edu; theo0027@umn.edu |
| Degree | Doctor of Philosophy |
| Document type | Dissertation |
| Degree program | Computer Science |
| School | Viterbi School of Engineering |
| Date defended/completed | 2011-01-11 |
| Date submitted | 2011 |
| Restricted until | Unrestricted |
| Date published | 2011-04-29 |
| Advisor (committee chair) | Schaal, Stefan |
| Advisor (committee member) |
Valero-Cuevas, Francisco Sukhatme, Gaurav S. Todorov, Emo Schweighofer, Nicolas |
| Abstract | Motivated by the limitations of current optimal control and reinforcement learning methods in terms of their efficiency and scalability, this thesis proposes an iterative stochastic optimal control approach based on the generalized path integral formalism. More precisely, we suggest the use of the framework of stochastic optimal control with path integrals to derive a novel approach to RL with parameterized policies. While solidly grounded in value function estimation and optimal control based on the stochastic Hamilton Jacobi Bellman (HJB) equation, policy improvements can be transformed into an approximation problem of a path integral which has no open algorithmic parameters other than the exploration noise. The resulting algorithm can be conceived of as model-based, semi-model-based, or even model free, depending on how the learning problem is structured. The new algorithm, Policy Improvement with Path Integrals (PI2), demonstrates interesting similarities with previous RL research in the framework of probability matching and provides intuition why the slightly heuristically motivated probability matching approach can actually perform well. Applications to high dimensional robotic systems are presented for a variety of tasks that require optimal planning and gain scheduling.; In addition to the work on generalized path integral stochastic optimal control, in this thesis we extend model based iterative optimal control algorithms to the stochastic setting. More precisely we derive the Differential Dynamic Programming algorithm for stochastic systems with state and control multiplicative noise. Finally, in the last part of this thesis, model based iterative optimal control methods are applied to bio-mechanical models of the index finger with the goal to find the underlying tendon forces applied for the movements of, tapping and flexing. |
| Keyword | stochastic optimal control; reinforcement learning,; robotics |
| Language | English |
| Part of collection | University of Southern California dissertations and theses |
| Publisher (of the original version) | University of Southern California |
| Place of publication (of the original version) | Los Angeles, California |
| Publisher (of the digital version) | University of Southern California. Libraries |
| Provenance | Electronically uploaded by the author |
| Type | texts |
| Legacy record ID | usctheses-m3804 |
| Contributing entity | University of Southern California |
| Rights | Theodorou, Evangelos A. |
| Repository name | Libraries, University of Southern California |
| Repository address | Los Angeles, California |
| Repository email | cisadmin@lib.usc.edu |
| Filename | etd-Theodorou-4581 |
| Archival file | uscthesesreloadpub_Volume14/etd-Theodorou-4581.pdf |
Description
| Title | Page 247 |
| Contributing entity | University of Southern California |
| Repository email | cisadmin@lib.usc.edu |
| Full text | were used to describe the kinematics of the hand trajectory (Harris & Wolpert 1998), (Todorov & Jordan 2002). In neuromuscular modeling, however, linear models cannot capture the nonlinear behavior of muscles and multi- body limbs. In (Li & Todorov 2004), an Iterative Linear Quadratic Regulator (ILQR) was first introduced for the optimal con-trol of nonlin- ear neuromuscular models. The proposed method is based on linearization of the dynamics. An interesting component of this work that played an influential role in the studies on optimal control methods for neuromuscular models was the fact that there was no need for a pre-specified desired trajectory in state space. By contrast, most approaches for neuromuscular optimization that use classical con-trol theory (see Section VI) require target time histories of limb kinematics, kinetics and/or muscle activity. In (Todorov 2005) the ILQR method was extended for the case of nonlinear stochastic systems with state and control dependent noise. The proposed algorithm is the Iterative Linear Quadratic Gaussian Regulator (iLQG). This extension allows the use of stochastic nonlinear models for muscle force as a function of fiber length and fiber velocity. Figure 6 illustrates the application of LQG to our arm model (Section II). Further theoretical developments in (Li & Todorov 2006) and (Todorov 2007) allowed the use of an Extended Kalman Filter (EKF) for the case of sensory feedback noise. The EKF is an extension of the Kalman filter for nonlinear systems. The has been only few examples of studies in the area of the biomechanics of the index finger which try to identity the underlying control signals for the case of movement and force production, either these signals corresponds to neural commands or tensions ap-plied on the tendons. More precisely on the experimental side, the work in (Venkadesan & Valero-Cuevas 2008b) investigated the neural control of contact transition between 233 |
