View Chapter

Chapter 35 — Multisensor Data Fusion

Hugh Durrant-Whyte and Thomas C. Henderson

Multisensor data fusion is the process of combining observations from a number of different sensors to provide a robust and complete description of an environment or process of interest. Data fusion finds wide application in many areas of robotics such as object recognition, environment mapping, and localization.

This chapter has three parts: methods, architectures, and applications. Most current data fusion methods employ probabilistic descriptions of observations and processes and use Bayes’ rule to combine this information. This chapter surveys the main probabilistic modeling and fusion techniques including grid-based models, Kalman filtering, and sequential Monte Carlo techniques. This chapter also briefly reviews a number of nonprobabilistic data fusion methods. Data fusion systems are often complex combinations of sensor devices, processing, and fusion algorithms. This chapter provides an overview of key principles in data fusion architectures from both a hardware and algorithmic viewpoint. The applications of data fusion are pervasive in robotics and underly the core problem of sensing, estimation, and perception. We highlight two example applications that bring out these features. The first describes a navigation or self-tracking application for an autonomous vehicle. The second describes an application in mapping and environment modeling.

The essential algorithmic tools of data fusion are reasonably well established. However, the development and use of these tools in realistic robotics applications is still developing.

Multisensor remote surface inspection

Author  S. Hayati, H. Seraji, B. Balaram, R. Volpe, B. Ivlev, G. Tharp, T. Ohm, D. Lim

Video ID : 639

Jet Propulson Lab, Pasadena, applies telerobotic inspection techniques to space platforms.

Chapter 32 — 3-D Vision for Navigation and Grasping

Danica Kragic and Kostas Daniilidis

In this chapter, we describe algorithms for three-dimensional (3-D) vision that help robots accomplish navigation and grasping. To model cameras, we start with the basics of perspective projection and distortion due to lenses. This projection from a 3-D world to a two-dimensional (2-D) image can be inverted only by using information from the world or multiple 2-D views. If we know the 3-D model of an object or the location of 3-D landmarks, we can solve the pose estimation problem from one view. When two views are available, we can compute the 3-D motion and triangulate to reconstruct the world up to a scale factor. When multiple views are given either as sparse viewpoints or a continuous incoming video, then the robot path can be computer and point tracks can yield a sparse 3-D representation of the world. In order to grasp objects, we can estimate 3-D pose of the end effector or 3-D coordinates of the graspable points on the object.

DTAM: Dense tracking and mapping in real-time

Author  Richard A. Newcombe, Steven J. Lovegrove, Andrew J. Davison

Video ID : 124

This video demonstrates the system described in the paper, "DTAM: Dense Tracking and Mapping in Real-Time" by Richard Newcombe, Steven Lovegrove and Andrew Davison for ICCV 2011.

Chapter 63 — Medical Robotics and Computer-Integrated Surgery

Russell H. Taylor, Arianna Menciassi, Gabor Fichtinger, Paolo Fiorini and Paolo Dario

The growth of medical robotics since the mid- 1980s has been striking. From a few initial efforts in stereotactic brain surgery, orthopaedics, endoscopic surgery, microsurgery, and other areas, the field has expanded to include commercially marketed, clinically deployed systems, and a robust and exponentially expanding research community. This chapter will discuss some major themes and illustrate them with examples from current and past research. Further reading providing a more comprehensive review of this rapidly expanding field is suggested in Sect. 63.4.

Medical robotsmay be classified in many ways: by manipulator design (e.g., kinematics, actuation); by level of autonomy (e.g., preprogrammed versus teleoperation versus constrained cooperative control), by targeted anatomy or technique (e.g., cardiac, intravascular, percutaneous, laparoscopic, microsurgical); or intended operating environment (e.g., in-scanner, conventional operating room). In this chapter, we have chosen to focus on the role of medical robots within the context of larger computer-integrated systems including presurgical planning, intraoperative execution, and postoperative assessment and follow-up.

First, we introduce basic concepts of computerintegrated surgery, discuss critical factors affecting the eventual deployment and acceptance of medical robots, and introduce the basic system paradigms of surgical computer-assisted planning, execution, monitoring, and assessment (surgical CAD/CAM) and surgical assistance. In subsequent sections, we provide an overview of the technology ofmedical robot systems and discuss examples of our basic system paradigms, with brief additional discussion topics of remote telesurgery and robotic surgical simulators. We conclude with some thoughts on future research directions and provide suggested further reading.

Reconfigurable and modular robot for NOTES applications

Author  Scuola Superiore Sant'Anna, The BioRobotics Institute (Pisa, Italy)

Video ID : 833

A surgical robot is built inside the abodominal cavity, thanks to a set of modules which are introduced orally and assembled inside the body.

Chapter 40 — Mobility and Manipulation

Oliver Brock, Jaeheung Park and Marc Toussaint

Mobile manipulation requires the integration of methodologies from all aspects of robotics. Instead of tackling each aspect in isolation,mobilemanipulation research exploits their interdependence to solve challenging problems. As a result, novel views of long-standing problems emerge. In this chapter, we present these emerging views in the areas of grasping, control, motion generation, learning, and perception. All of these areas must address the shared challenges of high-dimensionality, uncertainty, and task variability. The section on grasping and manipulation describes a trend towards actively leveraging contact and physical and dynamic interactions between hand, object, and environment. Research in control addresses the challenges of appropriately coupling mobility and manipulation. The field of motion generation increasingly blurs the boundaries between control and planning, leading to task-consistent motion in high-dimensional configuration spaces, even in dynamic and partially unknown environments. A key challenge of learning formobilemanipulation consists of identifying the appropriate priors, and we survey recent learning approaches to perception, grasping, motion, and manipulation. Finally, a discussion of promising methods in perception shows how concepts and methods from navigation and active perception are applied.

Combined mobility and manipulation - Operational space control of free-flying space robots

Author  Jeff Russakow, Stephen Rock

Video ID : 787

An environmental space is simulated in two dimensions using an air-bearing over a flat surface. The operational space-control framework enables the dynamically decoupled motion and force control of the object.

Chapter 55 — Space Robotics

Kazuya Yoshida, Brian Wilcox, Gerd Hirzinger and Roberto Lampariello

In the space community, any unmanned spacecraft can be called a robotic spacecraft. However, Space Robots are considered to be more capable devices that can facilitate manipulation, assembling, or servicing functions in orbit as assistants to astronauts, or to extend the areas and abilities of exploration on remote planets as surrogates for human explorers.

In this chapter, a concise digest of the historical overview and technical advances of two distinct types of space robotic systems, orbital robots and surface robots, is provided. In particular, Sect. 55.1 describes orbital robots, and Sect. 55.2 describes surface robots. In Sect. 55.3, the mathematical modeling of the dynamics and control using reference equations are discussed. Finally, advanced topics for future space exploration missions are addressed in Sect. 55.4.

DLR ROKVISS disassembly

Author  Gerd Hirzinger, Klaus Landzettel

Video ID : 336

This video shows the disassembly of the ROKVISS robot from the ISS by Russian cosmonauts who brought the manipulator back to the DLR at the end of its mission in 2011. It was indeed very valuable to be able to analyze the robot on the ground, after it had spent seven years in space.

Chapter 62 — Intelligent Vehicles

Alberto Broggi, Alex Zelinsky, Ümit Özgüner and Christian Laugier

This chapter describes the emerging robotics application field of intelligent vehicles – motor vehicles that have autonomous functions and capabilities. The chapter is organized as follows. Section 62.1 provides a motivation for why the development of intelligent vehicles is important, a brief history of the field, and the potential benefits of the technology. Section 62.2 describes the technologies that enable intelligent vehicles to sense vehicle, environment, and driver state, work with digital maps and satellite navigation, and communicate with intelligent transportation infrastructure. Section 62.3 describes the challenges and solutions associated with road scene understanding – a key capability for all intelligent vehicles. Section 62.4 describes advanced driver assistance systems, which use the robotics and sensing technologies described earlier to create new safety and convenience systems for motor vehicles, such as collision avoidance, lane keeping, and parking assistance. Section 62.5 describes driver monitoring technologies that are being developed to mitigate driver fatigue, inattention, and impairment. Section 62.6 describes fully autonomous intelligent vehicles systems that have been developed and deployed. The chapter is concluded in Sect. 62.7 with a discussion of future prospects, while Sect. 62.8 provides references to further reading and additional resources.

VIAC: The VisLab Intercontinental Autonomous Challenge

Author  Alberto Broggi

Video ID : 179

This is the official presentation of the VisLab Intercontinental Autonomous Challenge, the longest ever trip undertaken with driverless vehicles, from Italy to China. Four electric vehicles left Parma (Italy) on July 26, 2010, and arrived in Shanghai on Oct 28, 2010, after 3 months of driving and more than 15,000 km. Check www.viac.vislab.it for details.

Chapter 80 — Roboethics: Social and Ethical Implications

Gianmarco Veruggio, Fiorella Operto and George Bekey

This chapter outlines the main developments of roboethics 9 years after a worldwide debate on the subject – that is, the applied ethics about ethical, legal, and societal aspects of robotics – opened up. Today, roboethics not only counts several thousands of voices on the Web, but is the issue of important literature relating to almost all robotics applications, and of hundreds of rich projects, workshops, and conferences. This increasing interest and sometimes even fierce debate expresses the perception and need of scientists, manufacturers, and users of professional guidelines and ethical indications about robotics in society.

Some of the issues presented in the chapter are well known to engineers, and less known or unknown to scholars of humanities, and vice versa. However, because the subject is transversal to many disciplines, complex, articulated, and often misrepresented, some of the fundamental concepts relating to ethics in science and technology are recalled and clarified.

A detailed taxonomy of sensitive areas is presented. It is based on a study of several years and referred to by scientists and scholars, the result of which is the Euron Roboethics Roadmap. This taxonomy identifies themost evident/urgent/sensitive ethical problems in the main applicative fields of robotics, leaving more in-depth research to further studies.

Roboethics: Military robotics

Author  Fiorella Operto

Video ID : 775

Ethical, legal and societal issues in military robotics. The so-called field of military robotics comprises all the devices resulting from the development of the traditional systems by robotics technology: Integrated defense systems; and A.I. systems for intelligence and surveillance controlling weapons and aircraft capabilities. Unmanned ground vehicles (UGVs), or autonomous tanks: Armored vehicles carrying weapons and/or tactical payloads, intelligent bombs and missiles. UAVs (unmanned aerial vehicles): also referred to as autonomous flying vehicles (AFVs) or drones, unmanned spy planes and remotely piloted bombers. ASV (autonomous surface vessels) and patrol boats. AUVs (autonomous underwater vehicles): Intelligent torpedoes and autonomous submarines. In this field, the main problems could arise from: inadequate management of the unstructured complexity of a hostile scenario; the unpredictability of machine behavior; the increased risk of starting a video-game-like war, due to the decreased perception of its deadly effects; unpredictable side-effects on civilian populations; human-in-control hierarchy and robot’s transparency; psychological issues of humans in robotized environments (mixed teams); accountability and responsibility gap; the assignment of liability for misbehaviors or crimes. Collateral damages: Despite the increasing success of this technology, military hierarchies feel concerned about the potential dangers. Drones can accidentally fall and possibly damage humans and objects. Daily news report about unintended injury or death of innocent non-combatants (usually known as “collateral damage”) from war theaters. Potential friendly-fire casualties in crowded battlefield or due to enemy’s tracking/hijacking.

Chapter 69 — Physical Human-Robot Interaction

Sami Haddadin and Elizabeth Croft

Over the last two decades, the foundations for physical human–robot interaction (pHRI) have evolved from successful developments in mechatronics, control, and planning, leading toward safer lightweight robot designs and interaction control schemes that advance beyond the current capacities of existing high-payload and highprecision position-controlled industrial robots. Based on their ability to sense physical interaction, render compliant behavior along the robot structure, plan motions that respect human preferences, and generate interaction plans for collaboration and coaction with humans, these novel robots have opened up novel and unforeseen application domains, and have advanced the field of human safety in robotics.

This chapter gives an overview on the state of the art in pHRI as of the date of publication. First, the advances in human safety are outlined, addressing topics in human injury analysis in robotics and safety standards for pHRI. Then, the foundations of human-friendly robot design, including the development of lightweight and intrinsically flexible force/torque-controlled machines together with the required perception abilities for interaction are introduced. Subsequently, motionplanning techniques for human environments, including the domains of biomechanically safe, risk-metric-based, human-aware planning are covered. Finally, the rather recent problem of interaction planning is summarized, including the issues of collaborative action planning, the definition of the interaction planning problem, and an introduction to robot reflexes and reactive control architecture for pHRI.

Reach and grasp by people with tetraplegia using a neurally-controlled robotic arm

Author  Leigh R. Hochberg, Daniel Bacher, Beata Jarosiewicz, Nicolas Y. Masse, John D. Simeral, Jörn Vogel, Sami Haddadin, Jie Liu, Sydney S. Cash, Patrick van der Smagt, John P. Donoghue

Video ID : 618

The authors have shown that people with long-standing tetraplegia can use a neural interface system to move and click a computer cursor and to control physical devices. One of the study participants, implanted with the sensor five years earlier, also used a robotic arm to drink coffee from a bottle. Although robotic reach and grasp actions were not as fast or accurate as those of an able-bodied person, the results demonstrate the feasibility for people with tetraplegia, years after injury to the central nervous system, to recreate useful multidimensional control of complex devices directly from a small sample of neural signals.

Chapter 27 — Micro-/Nanorobots

Bradley J. Nelson, Lixin Dong and Fumihito Arai

The field of microrobotics covers the robotic manipulation of objects with dimensions in the millimeter to micron range as well as the design and fabrication of autonomous robotic agents that fall within this size range. Nanorobotics is defined in the same way only for dimensions smaller than a micron. With the ability to position and orient objects with micron- and nanometer-scale dimensions, manipulation at each of these scales is a promising way to enable the assembly of micro- and nanosystems, including micro- and nanorobots.

This chapter overviews the state of the art of both micro- and nanorobotics, outlines scaling effects, actuation, and sensing and fabrication at these scales, and focuses on micro- and nanorobotic manipulation systems and their application in microassembly, biotechnology, and the construction and characterization of micro and nanoelectromechanical systems (MEMS/NEMS). Material science, biotechnology, and micro- and nanoelectronics will also benefit from advances in these areas of robotics.

The electromagnetic control of an untethered microrobot

Author  Bradley J. Nelson

Video ID : 12

This is a video of a computer simulation showing the electromagnetic control of an untethered microrobot for ophthalmic applications, such as targeted drug delivery and epiretinal membrane peeling.

Chapter 20 — Snake-Like and Continuum Robots

Ian D. Walker, Howie Choset and Gregory S. Chirikjian

This chapter provides an overview of the state of the art of snake-like (backbones comprised of many small links) and continuum (continuous backbone) robots. The history of each of these classes of robot is reviewed, focusing on key hardware developments. A review of the existing theory and algorithms for kinematics for both types of robot is presented, followed by a summary ofmodeling of locomotion for snake-like and continuum mechanisms.

IREP tagging spikes

Author  Nabil Simaan

Video ID : 246

This video shows telemanipulation of the IREP (insertible robotic effectors platform). The IREP is a system having 21 controllable axes including two 7-DOF dexterous arms, 3-DOF camera head, an insertion stage, and two grippers [1]. Reference: [1] A. Bajo, R. E. Goldman, L. Wang, D. Fowler, N. Simaan: Integration and preliminary evaluation of an insertable robotic effectors platform for single port access surgery, Proc. 2012 IEEE Int. Conf. Robot. Autom. (ICRA), St. Paul (2012), pp. 3381-3387