Robot learning for disturbance-resilient control
Loading...
Date
2026-09-23
Authors
Δεληνικολής, Νικόλαος
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This thesis studies the swing-up and stabilization of an underactuated double pendulum in the Pendubot configuration: only the shoulder joint is actuated, subject to a severe torque limit of 0.15 N m, while the elbow swings freely. Because the elbow is unactuated, the upright configuration cannot be reached along quasi-static paths; the controller must steer the passive link dynamically through the inertial coupling between the links, injecting energy at a rate the torque budget tightly limits, arrive at the top nearly at rest, and then keep a lightly damped passive link balanced on an unstable equilibrium. Beyond nominal performance, the controller is required to be robust: the plant’s masses, lengths, centers of mass, and moments of inertia are randomized around their nominal values, and a single fixed policy must solve the task across the entire family of plants.
The problem is addressed with model-free deep reinforcement learning. A Soft Actor-Critic agent is trained with a thirteen-term, phase-structured reward whose distance-gated construction embeds the classical energy-based/balancing decomposition into a single differentiable objective, and with a multi-stage domain-randomization curriculum that enables physically meaningful parameter groups cumulatively, gated by strict windowed performance criteria with a hysteresis mechanism that certifies sustained rather than incidental competence.
Over a single two-million-step training run, the curriculum absorbed every randomization stage, and its per-stage costs form an order-dependent difficulty profile of the randomized physics under this schedule: mass randomization was absorbed almost for free, whereas inertia randomization of the passive link required twelve times the training budget. The final policy swings up from exact hanging rest into the 5 cm target zone in under one second, remains in-zone without interruption for the remainder of the evaluation episode, and satisfies the strict 1 cm stability condition shortly after entry, at residual velocities of order 0.01 rad/s. A post-hoc robustness sweep reveals that the learned robustness is organized along a diagonal compensation band in parameter space—mutually offsetting perturbations are absorbed even beyond the training support, while anti-compensated corners account for the curriculum’s residual failures—and controlled simulation ablations show the training-level observation noise acting as effective dither against limit cycles of the deterministic loop, which several off-nominal plants require for stabilization.
Collectively, the thesis demonstrates that a single learned feedback policy can replace the classical hybrid swing-up/balance architecture on a torque-limited underactuated plant, and it contributes a reusable methodological template: encode phase-dependent objectives through distance gating, schedule robustness demands behind demonstrated competence, and read the curriculum’s cost profile as an interpretable empirical signal about the problem under the chosen schedule.
Finally, zero-shot deployment of the same checkpoint on the physical Pendubot achieved 27/31 successful autonomous swing-ups—with a 0.57 s median tip-zone entry under the simulation-identical criterion—balance from manual placement held in all six trials with zero falls (median held duration 12 s; five of six exceeding the fixed 5 s criterion, one short recording censored), 34/35 recoveries from full knockdowns, and 37/38 recoveries from instrumented torque-pulse disturbances.
Description
Keywords
Machine learning, Robust-control, Double pendulum, Pendubot, Soft actor-critic, Curriculum learning, Sim-to-real