Using Intelligent Robots to Assemble Automobile Parts
Advances in  Automobile Engineering

Advances in Automobile Engineering
Open Access

ISSN: 2167-7670

+44 1300 500008

Review Article - (2017) Volume 6, Issue 1

Using Intelligent Robots to Assemble Automobile Parts

Weiwei Wan*
National Institute of Advanced Industrial Science and Technology (AIST), Japan
*Corresponding Author: Weiwei Wan, National Institute of Advanced Industrial, Science And Technology (AIST), Japan, Tel: +81-29-861-200 Email:


This paper presents the available technique and discusses the difficulties to implement intelligent robots in nextgeneration automobile assembly. First, it presents the status of automobile assembly line, analyzes the problems and difficulties of current industrial robots-based assembly systems, and summarizes the technology needed to overcome the problems and difficulties. Then, it presents the new technology for intelligent assembly developed at National Institute of Advanced Science and Technology (AIST), which can be directly used to implement intelligent robots for automobile assembly. Next, the paper analyzes the drawbacks of the newly developed technology and discusses remaining challenges, and presents our view of next-generation automobile assembly systems.


Keywords: Intelligent robots; Automobile assembly systems; Sensors; Databases

States in Automobile Assembly Line

Industrial robots are widely used in automobile assembly lines. These robots are called industrial robots since their goal is to follow some pre-taught paths or way points, which is essentially the same as automation machines. The tasks performed by these robots include welding, painting, pick-and-place in structured environment (e.g., glass installation, door installation, and nut/bolt fastening), etc.

The tasks performed by industrial robots are exciting. On the other hand, there are lots of tasks remained to be done by human workers. These tasks include (1) picking parts from cluster, (2) reorienting parts to certain poses, and (3) force-based assembly. These tasks cannot be done using industrial robots and teach pendants since: (1) The parts shapes and physical properties are varying. It is difficult to manually specify the grasping strategies for infinite number of parts. (2) The initial poses and goal poses of manipulated parts are changing. It is difficult to teach the robots all motions to reorient different parts. (3) The first two processes lead to variety in force control and assembly, which also make pre-teaching difficult.

One advisable way to automate the remaining tasks is to develop intelligent robots. The fundamental technique includes (1) computer vision for object recognition, (2) object analyzer for grasp planning, (3) motion planning, (4) force analysis and assembly planning, and (5) machine learning.

Development of Intelligent Robots for Assembly at AIST

The manipulation group at National Institute of Advanced Industrial Science and Technology (AIST), Japan, is developing intelligent robots to assembly objects. Over the years, we have developed software for all fundamental technique.

(1) We developed object recognition technique by using point clouds collected from structure light- based depth sensors. The developed platform has high precision as it uses point clouds and geometric constraints in the robot execution phase to avoid unexpected noises [1]. We also developed multi-view vision systems by considering the results of previous manipulation [2].

(2) We developed object analyzer and gripping planner by segmenting and clustering objects surface meshes, as well as considering torque and object surface properties. The planner can find candidate grasps for parallel industrial robotic grippers with probabilistic completeness. Some technical details are available in [3,4].

(3) We developed middle and low-level manipulation planners to generate robot motions [5]. In the middle level, we use regrasp and handover technique to compute the intermediate states of robotic grippers and objects, and generate pre and post robot configurations using motion primitives to orchestrate the computed intermediate states. In the low level, we use probabilistic motion planning algorithms like Rapidly-exploring Random Tree (RRT) to plan more detailed collision-free and kinematic available motion sequences between the intermediate configurations.

(4) We developed assembly planners by analyzing the constraints between objects [6]. The assembly planner could automatically generate assembly sequences (order of assembly), assembly directions, and available grasps. The assembly planner is a high-level planning component and could work together with middle and low-level planning components discussed in item (3), and the vision system discussed in item (1), to perform integrated assembly and motion planning.

(5) We are exploring cutting-edge technique like machine learning to solve difficult problems like picking from clutter [7]. The technique in items (1) to (4) was conventional and the goal was to find collisionfree and kinematic available robot and hand configurations. Instead of the conventional approach, we developed new picking solutions by taking advantages of obstacles. Our assumption was that collision might not be negative and might even be helpful. We used machine learning to learn how to take advantages of the obstacles.

(6) We analyzed the states of force sensors and used state machines to perform the last step of assembly. The details were published in [8] where a dual-arm robot assemble two objects connected using snap joints. Some similar technique is available in [9,10] which used force sensors and state machines to fasten bolts. This technique for object manipulation can be directly used by intelligent robots in automobile industry. Some of them have been implemented for the specific tasks. One example is to assembly exhaustive parts [11]. Conventional solution is human workers pick out components from feeding trays, place them down on fixtures, and command industrial robots to weld them. The industrial robots cannot be taught to perform the pick-andplace tasks as the components are at various poses. Using the newly developed visual detection and grasp and manipulation planning technique, intelligent robots take all roles. They detect the components and pick them out directly from the tray using vision technique. They reorient and place down the components on fixtures, and weld them, using grasping and manipulation planning technique. Human workers are not directly involved in the process. The technique is superior in that it is adaptive to non-structured environment. It has high precision vision based on low-cost structure light-based sensors, and has high success rate in planning by using dual-arm humanoid robots.


Although the fundamental technique has been developed, there remains several challenges. For one thing, each of the fundamental technique has some bottlenecks: (1) Structure light-based 3D vision is not applicable to non-lambert surfaces, non-reflective surfaces, and transparent surfaces. Parts with these surfaces widely exist in automobile assembly. Recognizing them remains challenging. (2) Probabilistically complete gripping planner is only applicable to parallel robotic grippers. It is difficult to plan complete grasps for robot grippers with varying finger shapes [12,13]. It is also difficult to plan complete grasps for robot hands with more than two fingers (although non-complete planners are available [14,15]. (3) Planning optimal sequences by considering many more constraints like energy consumption and by taking advantages of environmental structures like supporting pins [16] are computational infeasible. There could be infinite combinatorics which require fast pruning and carefully designed heuristics or heuristic database. (4) Collecting training data for realworld robots is difficult. Although some inspiring examples have been done in [7,17], it is still difficult to allow failures and program robots to learn from failures online in real factories. (5) Force sensing and state analyzing are highly dependent on specific objects. Automatically performing state analyzing requires allowance of failures, which is still lab work (as discussed in item (4).

For the other, it is difficult to integrate the available technique. The integration requires a large software platform spanning from computational geometry to force analysis and state machines. It also requires the support of databases. We have started a project named PyHiro at Github [18]. The goal of this project is to develop an integrated assembly system using the available technique. The platform is under the support of MySQL database to save robots, grippers, intermediate states, etc. Integrating and coordinating the roles of the various technique in the platform is an open problem.

Vision of the Next-generation Automobile Assembly System

Using the available technique, we could develop a highly automatic system where the input is an assembled object model (geometric information and physical properties) and the relative relations between the parts in the model, the output is a sequence of robot motions that pick out parts from boxes, reorient parts to specific poses, and assemble them. The robot analyzes the environment and decides whether to take advantages of environmental structures during the assembly process. It could also provide suggestions to factory managers about how to set up the environment to facilitate the assembly line.

The system will be suitable for varying parts and objects, especially for the assembly of the coming connected and autonomous cars, which have more complicated components to assembly. Compared with existing industrial robots, the system will further reduce the necessary human interventions and replace the human workers in nonteachable tasks with intelligent robots. The input to the system could be from CAD software or human demonstration. The system could be reconfigured quickly for different parts. The automobile parts produced by manufacturers will be directly supplied to the system without unboxing. The system computes the assembly sequences without human intervention for underlying details. This paper presented the technique and discussed the difficulties to implement this kind of system. We expect it will provide a study motivation for researchers in these technique and systems.


  1.  Wan W, Lu F, Wu Z, Harada K (2017) Teaching robots to do object assembly using multi-modal 3-D vision, Neurocomputing,.
  2. Harada K, Wan W, Tsuji T, Kikuchi K, Nagata K, et al. ( 2016) Iterative visual recognition for learning based randomized bin-picking, Proceedings of International Symposium on Experimental Robotics.
  3. Wan W, Harada K (2016) Developing and comparing single-arm and dual-arm regrasp. IEEE Robotics and Automation Letters 1.1: 243-250.
  4. Wan W, Harada K (2016) Achieving high success rate in dual-arm handover using large number of candidate grasps, handover heuristics, and hierarchical search. Advanced Robotics. 30: 1111-1125.
  5. Wan W, Igawa H, Harada K, Onda H, Nagata K, et al. (2016) A mid-level planning system for object reorientation. Available as ArXiv e-prints, eprint no.1608.03140.
  6. Wan W, Harada K, Nagata K (2017) Assembly sequence planning for motion planning, Assembly automation (Available as ArXiv e-prints, eprint no.1609.03108, 2017).
  7. Harada K, Wan W, Tsuji T, Kikuchi K, Nagata K, et al.( 2016) Initial experiments on learning-based randomized bin-picking allowing finger contact with neighboring objects, in IEEE International Conference on Automation Science and Engineering (CASE), pp. 1196–1202
  8. Rojas J, Harada K, Onda H, Yamanobe N, Yoshida E, et al. ( 2012) A relative-change-based hierarchical taxonomy for cantilever-snap assembly verification. Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 356–363.
  9. Aronson R, Bhatia A, Jia Z, Guillame-Bert M, Bourne D, et al.( 2016) Data driven classification of screw-driving operations. Proceedings of International Symposium on Experimental Robots.
  10. Nakamura A, Nagata K, Harada K, Yamanobe N (2014) Error recovery using task stratification and error classification for manipulation robots in physical distribution. Transaction on Control and Mechanical Systems, vol. 3, pp. 122–131.
  11. Harada K, Yoshimi T, Kita Y, Nagata K, Yamanobe N, et al. (2014) Project on development of a robot system for random picking - Grasp/manipulation planner for a dual-arm manipulator. Proceedings of IEEE International Symposium on System Integration (SII), pp. 583–589.
  12. Rodriguez A, Mason MT (2012) Grasp Invariance.International Journal of Robotics Research.
  13. Rodriguez A, Mason MT (2013) Effector form design for 1DOF planar actuation. Robotics and Automation (ICRA), IEEE International Conference on. IEEE.
  14. Ciocarlie M, Allen P (2009) Hand posture subspaces for dexterous robotic grasping. International Journal of Robotics Research 28: 851–867.
  15. Hang K, Li M, Stork JA, Bekiroglu Y, Pokorny FT, et al. (2016) Hierarchical fingertip space: A unified framework for grasp planning and in-hand grasp adaptation. IEEE Transactions on Robotics, pp. 960–972.
  16. Cao C, Wan W, Pan J, Harada K (2016) Analyzing the utility of a support pin in sequential robotic manipulation. Proceedings of IEEE International Conference on Robotics and Automation.
  17. Levine S, Pastor P, Krizhevsky A, Quillen D (2016) Learning hand-eye coordination for robotic grasping with large scale data collection. Proceedings of International Symposium on Experimental Robotics.
  18. A Python Platform to Control the Hiro/Nxo/Hrp5p Robots. Available at,
Citation: Wan W (2017) Using Intelligent Robots to Assemble Automobile Parts. Adv Automob Eng 6:160.

Copyright: © 2017 Wan W. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.