Quality aspects are of great importance since they make a big impact on the final price of the product, apart from improving the Companies’ image. The automobile industry is not indifferent, and every day improvements are introduced in its quality processes. The cars sheet quality analysis is one of the features in which a significant progress is being made [1]. The main objective is to work in an automatic system that provides the quantification and classification of the imperfections on the sheets of the auto bodywork, due to the squeezing process [2]. Currently, this classification is done by the direct observation of the pieces.

The final goal is to obtain a classification that is as similar as possible to the one obtained by visual inspection. Figure 1 show all the items involved in the classification problem: data acquisition, analysis, classification, and decision for acceptability criteria. This work is done under collaboration between Carlos III University and PSA Peugeot and is under patent [3].

Figure 1: Quality control system.

Aspect imperfections on stamped parts can be generated through siderurgical processes or through forming process operations (deep drawing). In the elaboration of the steel coil, the imperfections can appear during the hot and cold rolling operations and during the galvanization or electro galvanization processes [4]. The main imperfections are: shell, spreading’s, oxides, red scales, white spots, scratches, drosses, and coating ripples (rolling direction, pick up, nozzle streaks, and the like).

The steel coil is then cut in order to obtain blanks. This operation can generate metallic dust or particles. This pollution will create punctual imperfections during the deep drawing operations that can be classified as:

Punctual imperfection (0.2 mm depth, 2 mm length (diameter)). These imperfections are created by an external element (a metallic particle, for example). During the contact between the tool and the blank, a punctual imperfection will appear. The external element either evacuates with the part or stays in the tool and generates other imperfections. These kinds of imperfections are non-repeatable (or on a small number of parts).

During the squeezing operation, aspect imperfections such as holes, bumps, and undulations can appear around the deformed zones. These imperfections are due to a local modification of the shape (around a door handle, for example) which creates a stress gradient and a deformation. These kinds of imperfections are repeatable.

The present paper deals with the identification of the last kind of imperfections (bumps) in order to characterize and evaluate them. The main problem to be solved in this work is the automatic evaluation of these imperfections, generally consisting on bumps of 10 m to 200 m height and 50 mm to 200 mm width.

Recent studies can be found in literature related to the superficial analysis in the automobile industry [5,6]. Among these systems, there are those using an image, such as fringe projection systems, dark field scattering [7] or retro reflection-based systems. Fringe projection systems consist on projecting parallel fringes or regular patterns on the surface of an object, and taking measurements of the deformation of the lines or patterns, or measurements of the phase shift produced by the surface in the pattern signal, when using interferometry. This technique has been used for the reconstruction of 3-D surfaces in medicine [8] and in industrial applications [9].

Among these systems, we can find the ATOS system [10], whose operation is based on stereo-vision. The system projects different kinds of fringes on an object. The fringes are captured by two CCD cameras. By taking measurements of the fringes deformation, the system re-builds the 3-D shape. The system is accurate for the threedimensional reconstruction of big objects, with a 0.5 mm height precision. However, for imperfections from 10 m to 200 m height (our case) it does not perform properly. Another system is the Opto TOP system [11], which consists of an emitter of different fringes and a CCD camera. The patterns appear on the object as dark and light lines. The 3-D image is calculated by triangulation. This system has around 10 m height precision, but the scanning areas are small and too much time is needed to process the whole image. Another important system based on fringe projection is the ONDULO system [12]. It produces a visual highlight of the imperfection, although the image has no superficial numeric quality.

Retro reflection systems [13-16] are based on the projection of white light on the surface to be analyzed. The light is reflected from the surface to a screen and it is reflected again to the camera through the surface. The system highlights local changes in the gradient of the surface, such as dents or undulations. The basis of the method consists on choosing properly the illumination system [17]. Patent [18] describes an optical system to get the retro reflective images and patent [19] extends optical configurations for the system. Patent [20] uses a system for the inspection of a whole vehicle using several light focuses and several cameras that capture the reflected light from the metal surface, doing the surface analysis by triangulation. Patents [21] and [22] use optical devices but only in painted surfaces. The main differences between these patents and our work are the hardware configuration of the system and the algorithm to process the image.

Finally in order to get the classification, several methods are used in the literature such as KNN [23], Naive Bayes [24], SVM [25] or fuzzy logic [26] classifiers can be used in this work, showing good results in classification tasks [27]. The system proposed in this work uses a spot lighting source, a motorized device to place the metal sheets in the optimal position for the image acquisition, an algorithm for imperfection evaluation that extracts geometrical information from the pro les of the image, and a classifier of the severity of the imperfection from these parameters. The final result of the full system is not reached in the mentioned previous works.

The complete system that leads to the imperfection classification from the sheet is shown in Figure 2, and consists of:

Figure 2: From metal sheet to quality index.

Image acquisition: to establish the classification, the first step is to extract the geometrical characteristics of the sheet from an image. This image is taken by a system consisting of a motorized table, a light source with an optical fiber guide, a motorized camera, and a screen.

Determination of the parameters: an algorithm has been implemented that allows extracting from the images the parameters from which the geometrical properties of the sheet are determined. These parameters are related to the quality of the sheet, and the classification can be established from them.

Determination of the quality index: from the geometrical information obtained by the algorithm and the rules provided by the visual experience on the criticity of the imperfections, a criticity index is determined for that imperfection.

The process starts with the appropriate optical system that enhances the imperfection. The acquired image must contain features and data that allow getting quantitative measurements. This step is critical in order to obtain useful information from the imperfection. The second step of the system consists of the parameter obtention. An algorithm has been implemented [28] in order to extract the geometric characteristic of the imperfection. These parameters are related to the quality of the sheet, and are the input for the proposed classification system. Finally a classification criterion has been developed from the previous data that leads to the quality index that is similar to the previous one performed by quality human expert. Among several classifiers, a fuzzy classifier has been finally selected.

The image system is based on the light retro reflection property. With this property of the light, deformations that the human eye is not able to detect can be enhanced (Figure 3). To get the adequate image, a system has been developed for detecting imperfections in the sheets (Figure 4). It consists of:

Figure 3: Imperfections enhancing process.

Figure 4: Full system.

An image acquisition system: it includes a spot lighting source focusing on the imperfection, which produces the brightness for enhancing the defects. The light is guided to the optimum position and orientation through optical fiber. A camera, a video conversion device and a retro reflective screen complete the image acquisition system.

A motorized device: consist of a mobile structure, in which the sheet to be analyzed is placed over it, with a shaft driver that controls the (x, y) position of the table and the z position of the camera. The system includes adaptive joints in order to x component positions of the device and the camera, allowing the studies of repeatability in the capture and the subsequent analysis. A personal computer in which the control of the motorized device and the camera is performed. In this computer image analysis and classification algorithms are performed too.

The classification system that wants to be automated uses parameters of imperfections that can affect the visual appearance of the vehicle, when those imperfections are very severe. Currently, experts believe that a weighting be-tween the height of the defect, its length and its area are the most significant parameters to evaluate the extent of the defect.

Considering as starting point the image obtained from the retro reflexion system, an algorithm has been designed [28]. This algorithm calculates the height of the defect, its width and length from the profile information of the post processed images (Figure 5).

Figure 5: Parameters obtention from the image.

For the estimation of the parameters, the relationship between the points of the pro les, obtained from the retro flexion image of retro flexion and the real pro les, are taken into account, as shown in Figure 6.

Figure 6: Relations from retroreexive image and metal sheet.

The points of inflection of the obtained pro le correspond to points of the real pro le of zero derivatives, and set the points of start-end and maximum height of imperfection.

The obtained maximum point of the pro le corresponds to the point of maximum slope of the real profile. Considering this information, the defect to be analyzed is chosen and an algorithm is run along the defect, checking their profiles. The algorithm provides the start and end points, and the maximum gradient points for each profile. With this set of points, the length of the defect, the defect area and its maximum gradient are calculated. These parameters are the input to the classification system.

Our objective is to develop a system that classifies the imperfections according to the criticity criteria currently used by the experts on the subject. For that purpose, the following requirements must be fulfilled:

• To use a natural language, similar to the experts’ one.

• To handle uncertainties.

• To reflect the rules used by the experts.

• To produce qualitative information.

Taking into account these considerations, a classifier based on fuzzy logic [26] has been closed, instead of other classifiers such as KNN, Naive Bayes or SVM, due to the usability for the experts. The fuzzy logic is shown as a technique to represent the vagueness and imprecision of the concepts used in natural language. It is based on the fuzzy sets, which were defined as an extension of the classic sets capable of modelling the imprecision proper of the human concepts. They work with variables defined by qualitative labels (large small) that are associated with numerical values that do not need to be precise. To calculate the values of the output variables, also qualitative, a set of rules are used to establish the relationship between the input and output variables.

The developed system calculates as an output a criticity index for the imperfection, taking as inputs the geometrical parameters calculated by the algorithm previously explained and using a set of rules that relate the geometrical parameters to the quality.

The classification system has been developed taking into account that:

The inputs are the geometrical parameters of the imperfection taken into account in production: length, width, and gradient. It must manage qualitative values: small, medium, large.

The relationship between qualitative values and mathematical values is set according to the quality parameters specified by the experts, using mamdani logic [26].

Input variables

The input variables to be considered for the classifier are the length in millimeters, the width in millimeters, and the gradient of the imperfection (gray variation per millimeter). Several qualitative membership functions are considered for each variable, for instance: small, medium, large. The relationship between the qualitative values and the mathematical values are set by the experts for each input variable. Figure 7 shows examples of membership functions.

Figure 7: Examples of membership functions.

Output variables

The output variable of the classifier is the criticity index. This index must take a qualitative value according to the four levels of quality assigned by the experts, labeled with the tags 0, 3, 5, and 10, similar to the tags used in production. Therefore, it is necessary to de ne membership functions for each quality value of the output. This way, the system provides a qualitative value of quality according to the rules defined.

Decision rules

Once the variables of the system have been defined, we must set the rules that establish the level of quality from the geometrical properties of the imperfection. These rules have been developed in collaboration with experts and in accordance with the standards set. The rules are set using a natural language. For example:

“If the imperfection is severe, with large length and small width, the criticity index is high.” They are set by the experts and are based on the final visual effect that the imperfection causes on them, according to their experience.

First of all, the system transforms the numerical input parameters from the geometry of the imperfection (gradient, length, and width) into qualitative values, absorbing the imprecision of such numerical values. Then, all the decision rules are evaluated and the qualitative value of the output is calculated basing on the output membership functions. These membership functions are the different values that the criticity index can take. Figure 8 shows how the system performs all these functions, evaluating the rules for the variables involved in the process.

Figure 8: Performance of the fuzzy logic system.

This section describes the performance process of the system, from the preparation of the system to the defect classification.

System preparation

220 The door of the car that is going to be evaluated is mounted in the motorized system; see Figure 9, so the defect is within the working area of the optics system. The lighting system has been previously tuned to the work environment.

Figure 9: Sheet placed in the inspection system.

Through the controller’s axes the sheet is placed in such a way, that the direction of the optical system will generate the maximum contrast in the defect. After a process of calibration of the system, the defect to be analyzed is marked and the automatic process of evaluation and classification of the defect is launched.

Calibration of the system

The calibration of the system consists on determining the ratio between a pixel and its value in millimeters. To solve this problem, the following method is used. Two circumferences of known diameter are placed on the sheet, introducing their center and radius to the system. The system calculates automatically the border (in pixels) of the circumferences by morphological operations based on a dilatation and an erosion of the image to eliminate the noise and by a Sobellter to extract the borders.

The calibration system stores a binary mask with the borders of the circumferences. During any stage of the algorithm, the ratio between pixels and millimeters can be calculated in two stages:

1. For each direction, the ratio between pixels and millimeters is determined establishing the proportionality between the value of the diameter in millimeters introduced in the system and the diameter in pixels of the circumference in that direction.

2. The factor of proportionality is weighted by the distance from the pixel to the centers of the circumferences to avoid the measurement scaling problem due to the image depth.

In Figure 10 the calibration system tool is presented. All tools have been developed in Matlab.

Figure 10: Calibration system tool.

After executing the algorithm, and taking into account the calibration sys-tem, the software developed shows the graphical and numerical results obtained from this procedure (Figure 11).

Figure 11: Interface with graphical and numerical results obtained from the algorithms.

In addition to the numerical results of the imperfection, the following information is also obtained: Maximum gradient: it is the highest value of all the gradients. It indicates the severity of the imperfection.

Direction of the maximum gradient: it indicates the direction in which the imperfection is the most visible.

Imperfection line: it is formed by the union of the points of maximum gradient. This line covers the most prominent area of the imperfection, showing its length visually.

Length of the imperfection: it is determined by the addition of the distances between the points of maximum gradient.

Width of the imperfection: it is the longest distance between the start and end points of each profile. Area of the imperfection: it is calculated by integrating the area formed by the curve that joins the start and end points of each profile.

To test the validity of the quantification and classification system, its whole performance has been checked, from the image capture process to the determination of the Quality index. Different statistic results are also shown from the experimental tests carried out on car doors in the production process at the factory.

Preparing test

The whole system must place the metal sheet, adjust the camera optics, and run the algorithm taking into account that the production time for this process in the factory is around 1 min and 23 s. The classification process starts with the capture of the image of the imperfection to be evaluated. For the operation of the whole system, an application has been developed from which the motorized table, the image capture process, and the calculation of the parameters and the Quality index can be managed. First, after placing the table and the camera in the proper position by the hardware monitoring system, the image capture process is carried out. Once the system is calibrated, the imperfection to be analyzed is selected and its corresponding geometrical parameters are calculated. After running the algorithm for the evaluation of the imperfection, the results shown in Figure 12 are obtained. After the determination of the geometrical parameters, the data le of the fuzzy classifier previously defined is loaded into the system, which contains the membership functions of the input and output variables and the rules defined by the experts for the classification. The Quality index can take four qualitative tags: 0, 3, 5, and 10. The tag 0 corresponds to a negligible imperfection, while the tag 10 corresponds to a very significant one.

Figure 12: Evaluation results.

Validation tests

Once the correct performance of the system has been checked, different experimental tests have been carried out for several imperfections located in different car doors coming from the production process, in the laboratory. Figure 13 shows the geometrical parameters obtained for the first 10 doors tested, together with the criticity index determined by the experts and by the system. Only one of the values differs from the one assigned by the experts.

Figure 13: First 10 tests for validation in the laboratory.

Finally, the system has been tested on line, in the production line, for the evaluation of a batch of 40 car doors. Though the criticity indexes obtained automatically for several imperfections differ from those obtained by visual inspection, as shown in Figure 14, the later analysis and discussion about the imperfections in the laboratory validate the indexes assigned by the quantification and classification system (see the “Final decision” column in Figure 14).

Figure 14: Validation tests performed in the production line.

This paper has presented a system for the characterization of the imperfections produced in the sheets composing the auto bodywork due to the squeezing process in an automatic way. The system consists of a motorized table, a retro reflective screen, a spot lighting source, a camera, and a computer that processes the images obtained. An algorithm based on the extraction and filtering of the pro les has been developed for the determination of the maximum gradient point, related to the height of the imperfection, and the points that delimit its width. A fuzzy logic-based classifier has been also developed for the calculation of the Quality index of the imperfection from its geometrical parameters and the classification rules provided by the experts. Fuzzy logic presents as main advantage the usability for experts. Experimental results validate the operation of the quantification and classification system in the production line, showing good results comparing with the decisions of the quality experts.

The authors gratefully acknowledge the funds from PSA PEUGEOT-CITROEN automobile group. Special thanks also to Jose Manuel Pequeno, for his support along this project and to Yannick Bertret and Vincent Nalewajk, for the assistance during the work.