Multi-axis robot arms perform a wide variety of processes in automotive assembly plants, including painting, welding and fastening.
However, even with advances in automation technology, some processes still cannot be completed without skilled human assemblers. The task of installing wire harnesses into car bodies is one such task that has traditionally been difficult for robots.
There has been some previous research related to the problems of handling deformable linear objects, such as wire or tubing, with robots. Many of these studies concentrated how to deal with topological transition of deformable linear objects. They tried to program robots to tie knots or create loops with rope. These studies applied mathematical knot theory to describe the topological transitions of the rope.
In these approaches, a deformable linear object in three dimensions is first projected into a two-dimensional plane. The projection in the plane, which is demonstrated as crossed curves, can be well-described and treated using knot theory.
In 2006, a research team led by Hidefumi Wakamatsu, Ph.D., of Osaka University in Japan developed a method for knotting and unknotting deformable linear objects with robots. They defined four fundamental operations (among them, three are equivalent to Reidemeister moves) necessary for completing a transition between any two wire-crossing states. The researchers showed that any knotting or unknotting operation that can be decomposed into sequential topological transitions can be achieved by employing a sequential combination of these four fundamental operations. Their approach was verified when they were able to program a SCARA robot to knot a rope placed on a desk.
Similarly, researchers led by Takayuki Matsuno, Ph.D., of Toyama Prefectural University in Imizu, Japan, developed a method for knotting a rope in three-dimensions using two robot arms. One robot held the end of the rope, while the other knotted it. To measure the three-dimensional position of the rope, stereo vision was employed. The state of the knot is described using knot invariants instead of Reidemeister moves.
In both studies, the robots were equipped with a classic, two-fingered parallel gripper with only one degree of freedom.
In 2008, a research team led by Yuji Yamakawa of the University of Tokyo demonstrated a technique for knotting rope using a robot equipped with a high-speed multi-fingered hand. With a more dexterous gripper—including force and torque sensors mounted in the fingers—operations such as “rope permutation” become possible, even with one arm. Rope permutation refers to the operation of exchanging the places of two ropes by twisting them while pinching the ropes between two fingers.
Other research projects have focused on solving problems related to robotic handling of deformable linear objects on the assembly line.
For example, Tsugito Maruyama, Ph.D., and a team of researchers at Fujitsu Laboratories Ltd. in Kawasaki, Japan, developed a wire-handling system for an assembly line making electrical parts. A robot arm was used to insert signal cables into clasps. Two technologies were critical for enabling their system to work: a multi-planar laser light projector and a stereo vision system.
Jürgen Acker and researchers at Kaiserslautern University of Technology in Germany developed a method for using 2D machine vision to determine where and how a deformable linear object (in this case, an automotive cable) contacts objects in the environment.
Based on all this research, we attempted to develop a practical robotic system for installing wire harnesses on an automotive assembly line. Although our system was developed in the laboratory, all the conditions employed in our experiments are referenced from a real automobile plant. Our goal was to demonstrate the technical feasibility of such a system and determine areas where further development is necessary.
Wire Harness Assembly
An automotive wire harness consists of multiple cables wrapped by electrical tape. It has a tree-like structure with each branch connected to a specific instrument. On the assembly line, a worker manually attaches the harness to the instrument panel frame.
A set of plastic clamps are bound into the wire harness. These clamps correspond with holes in the instrument panel frame. Attachment of the harness is achieved by inserting the clamps into the holes. A robotic system for installing the harness must therefore solve two basic problems: how to measure the state of a wire harness, and how to handle it.
A wire harness has complex physical properties. During assembly, it exhibits both elastic deformation and plastic deformation. This makes obtaining a precise dynamic model of it difficult.
Prototype System
Our prototype harness assembly system consists of three, compact six-axis robots positioned in front of an instrument panel frame. The third robot helps with positioning and grasping the harness.
Each robot is equipped with a two-fingered parallel gripper with one degree of freedom. The gripper fingers have two indentations: one for holding the harness clamps, the other for holding segments of the harness itself.
Each end-effector is also equipped with two CCD cameras and a laser range sensor. The two cameras have different focal lengths to provide a large depth of field. The laser range sensor is used when precise measurement to a wire segment is necessary. Surrounding the workcell, 10 additional fixed-position cameras face the work area from various directions. Including the cameras mounted on the end-effectors, our system employs a total of 16 vision cameras.
Recognition of the harness is accomplished with machine vision. A specially designed plastic cover is attached to each harness clamp. The covers have geometric patterns that are read with ARToolKit software. This open-source software was originally designed for augmented reality applications. It provides a set of easy-to-use libraries for detecting and recognizing the markers. The camera reads the markers to determine the relative position of the harness.
Each clamp cover has its own geometric pattern. The pattern tells the robot controller the relative position of the harness in space, as well as information regarding that segment of the harness (such as where that segment should be positioned on the panel frame).
The fixed cameras around the workcell provide rough positional information about each harness clamp. The position of a specific harness clamp is estimated by interpolating the position of adjacent clamps. The end-effector is guided to approach to the target clamp with positional information obtained from the fixed cameras—until the wrist camera can find the target. From that moment, robot guidance is provided solely by the wrist camera. The precision provided by the wrist camera in that short distance ensures reliable grasping of the clamps.
A similar process is used to grasp a deformable segment of the wire harness. The position of the target segment is first estimated by interpolating the pose of adjacent clamps. Since the interpolated curve is not precise enough to guide the robot, the estimated area is then scanned by the laser scanner. The scanner emits a planar beam with a particular width. The exact position of the segment can then be determined from the distance profile obtained from the laser sensor.
The markers greatly simplify measurement of the wire harness. Although the clamp covers increased the cost of the system, they greatly improve the reliability of the system.
Harness Handling
The harness clamp is designed to mate with a hole in the panel frame. Thus, the gripper grasps a clamp by its base and inserts its toe into the hole.
In addition, there are some occasions in which it is necessary to handle a wire segment directly. For instance, in many processes, one robot must shape the harness before another robot can perform its job. In such a case, one robot needed to orient a clamp so that it could be reached by another robot. The only way to do this was to twist a nearby wire segment.
Initially, we attempted to shape the wire by twisting its adjacent clamp. However, due to the low torsional stiffness of the wire segment, this proved to be impossible. In subsequent experiments, the robot gripped and bent the wire segment directly. During this process, pose of the target clamp is monitored by the surrounding cameras. The bending process will continue until the orientation of the target clamp coincides with a reference value.
Verification Experiments
Once we developed a prototype assembly system, we ran a series of experiments to test it out. The process starts with the robots picking up a wire harness from a hanger. They then insert eight harness clamps into the panel frame. The process ends with the robots returning to the initial standby position.
The right arm inserts clamps 1, 2 and 3. The central arm inserts clamps 4 and 5, and the left arm inserts clamps 6, 7 and 8.
Clamp 3 in inserted first, followed by clamps 1 and 2. Clamps 4 through 8 are then inserted in numerical order.
The motion sequence of the robot arms was generated using simulation software. A collision detection algorithm prevented the robots from knocking into objects in the environment or each other.
In addition, some operations in the motion sequence were generated by referencing human assemblers. For this purpose, we captured the motions of workers during assembly. The data include both the motion of the worker and the corresponding behavior of the wire harness. Not surprisingly, the motion strategy taken by a worker often proved to be more effective than that of the robots.
Twisting Control of Wire Segments
In our experiments, we sometimes ran into difficulties inserting the clamps because it was impossible to position the gripper for the task. For example, clamp 5 should be inserted immediately after clamp 4 has been fixed to the frame. However, the harness segment left of clamp 4 would invariably droop, making it difficult for the center robot to position clamp 5 for insertion.
Our solution to this problem was to pre-shape the target wire segment to ensure a successful grasping. First, clamp 5 is raised up by the left robot by gripping the wire segment near clamp 5. Then, the orientation of clamp 5 is regulated by controlling the torsional state of the wire segment. This pre-shaping operation ensures that the subsequent gripping of clamp 5 is always executed in the most appropriate position.
Cooperation Between Arms
In some situations, assembly of a wire harness requires human-like cooperation between multiple robot arms. Insertion of clamp 1 is a good example. Once clamp 2 has been inserted, clamp 1 will droop. The space available to insert clamp 1 is limited, and it is difficult to position the gripper due to the risk of colliding with the surrounding environment. Moreover, practical experience taught us to avoid starting this operation with that segment of the wire drooping, since it could lead to wire segments being caught by the surrounding frame in subsequent operations.
Our solution to this problem was inspired by the behavior of human workers. A human worker easily coordinates the use of his two arms to complete a task. In this case, a worker would simply insert clamp 4 with one hand, while simultaneously adjusting the position of the wire segment with the other hand. We programmed the robots to implement the same strategy.
Plastic Deformation
In some situations, it was difficult to pre-shape the wire segment by cooperatively employing two robots. The process of inserting clamp 6 is a good example. For this operation, we expected that the left robot arm would insert it into the frame, since it is the only robot arm that can reach the target.
As it turned out, the robot could not initially reach the clamp. When the controller determines that gripping the clamp is not achievable, the robot will try to grip the wire segment near the clamp instead of gripping the clamp itself. The robot then twists and bends the segment to turn the clamp face more to the left. Bending a segment a few times is usually enough to change its position. Once the segment is an appropriate position for gripping, the robot will make another attempt to grip the target clamp.
Conclusions
Ultimately, our robotic system was able to install eight clamps into the instrument panel frame with an average time of 3 minutes. Although that speed is still far from the requirement for practical application, it does demonstrate the technical feasibility of robotic wire harness assembly.
Several problems must be solved to make the system reliable and fast enough for practical industry application. First, it’s important for wire harnesses to be pre-shaped for robotic assembly. Compared with knotting and unknotting operations, the torsional state of individual wire segments is critical for wire harness installation, since the robots are handling parts bound into the harness. In addition, a gripper equipped with twisting degree of freedom would also help with harness installation.
To improve the speed of the process, the dynamic behavior of the wire should be considered. This is evident in the film studies of skilled workers inserting wire harnesses. They use both hands and skilled motion to control the dynamic swinging of the wire and thereby avoid surrounding obstacles. When implementing robotic assembly with similar speed, special approaches will be necessary to suppress the dynamic behavior of the wire.
Though many of the approaches employed in our research are straightforward, we successfully demonstrated automatic assembly with our prototype robotic system. There is potential for automation with these kinds of tasks.
Author’s note: Kyong-mo Koo, Ph.D., assistant professor; Kohei Kikuchi, Ph.D., assistant professor; Atsushi Konno, Ph.D, associate professor; and Masaru Uchiyama, Ph.D., professor; were part of the research team that developed the workcell. The work was funded by the NEDO Project for Strategic Development of Advanced Robotics Elemental Technologies. Our prototype system was designed with reference to the assembly line of Suzuki Motor Corp. under a cooperation agreement.
ASSEMBLY ONLINE
For more information on automated wire harness assembly, visit www.assemblymag.com to read these articles:
