Abstracts Computer Science

Sensor Fusion for Closed-loop Control of Upper-limb Prostheses

by Marko Markovic

This thesis addresses open challenges in the field of myoelectrically controlled upper limb prostheses. These challenges include the inherently low bandwidth of the myoelectric control channel that makes the current control interfaces limited and unintuitive for the user, especially when controlling modern multi-function prostheses, as well as the lack of somatosensory feedback that would allow the user to better perceive the state of his/her prosthesis. This thesis aims at addressing these challenges by designing novel man-machine-interfaces, based on the latest sensing and automatic control technologies, to provide improved operation and perception of the prosthetic device. To this end, the thesis comprises introductory chapters that describe the state of the art in the field, the aim of the thesis and the used methodology, as well as four peer-reviewed journal publications presenting the novel feedforward and feedback methods. In the first two studies, I proposed and evaluated a novel system for prosthesis control based on sensor-fusion. In the classic approach, the user has the responsibility of generating all the command signals, while the prosthesis controller operates as a decoder, acquiring the signals and decoding user intention. In the novel framework, proposed here, the prosthesis is enhanced with advanced sensing and autonomous decision-making, becoming thereby an intelligent agent assisting the user. The inspiration for this novel approach comes from the modern-day autonomous robotic systems that utilize a variety of multimodal sensors and data processing methods in order to perceive and interpret the environment. In the present work, the prosthetic hand was equipped with computer vision and inertial sensing, and this information was used by the prosthesis controller to provide an additional, artificial-intelligence processing layer. This component analyzed the usage-context (environment, user and prosthesis) and, based on this, automatically controlled the hand preshape and orientation, thereby supporting the user in operating the prosthesis functions. The overall control loop is therefore simplified for the user because the sensor-fusion controller takes over the part of the inherent control complexity by adjusting the prosthesis parameters automatically. The user only provides high-level commands (e.g., grasp an object) that can be delivered robustly through a simple two-channel myoelectric interface. In the second two studies, I introduced a versatile development framework for evaluating a variety of feedback interfaces. The framework comprises a library of components implementing specific elements of a generic closed-loop prosthesis control system, from control inputs to the feedback interfaces. The framework operates in real-time and allows fast prototyping and testing. This has been used to develop and evaluate a novel biofeedback paradigm that closes the loop by feeding the myoelectric control signals (prosthesis input) back to the user. This is a novel approach with respect to the classic… Advisors/Committee Members: Farina, Dario (advisor), Graimann, Bernhard (referee), Sax, Ulrich (referee).