KR102454452B1 - Method, device and system for processing reverse engineering of car body structure using 3d scan data - Google Patents

Abstract

According to one embodiment of the present invention, a method for processing reverse engineering of a vehicle body structure using three-dimensional (3D) scan data, which is performed by an apparatus, comprises: a step of acquiring first scan data, 3D data generated by scanning a plurality of components from a 3D scanner, respectively, when the plurality of components are scanned by the 3D scanner, respectively, in a state that a first vehicle is disassembled into the plurality of components; a step of analyzing which component is scanned to generate each of a plurality of pieces of scan data included in the first scan data; a step of checking a first point, a location where a first component is installed in the first vehicle, when it is analyzed that 1-1 scan data in the first scan data is generated by scanning the first component; and a step of performing reverse engineering of a vehicle body structure around the first point based on the 1-1 scan data. Accordingly, the present invention has an effect of processing reverse engineering of a vehicle body structure in a convenient and accurate manner by using 3D scan data.

Description

Method, device and system for reverse engineering body structure using 3D scan data {METHOD, DEVICE AND SYSTEM FOR PROCESSING REVERSE ENGINEERING OF CAR BODY STRUCTURE USING 3D SCAN DATA}

The following embodiments relate to a technique for reverse engineering a body structure using 3D scan data.

Scan data using a 3D scanner is increasingly widely used in the fields of geometric modeling and medical morphology. The technology using 3D scan data can also be applied to a method of checking the results of a process for manufacturing a specific product. 3D scan data may be used to verify that the manufactured product is made as designed.

Most 3D scanners can scan a target object from various angles and generate scan data. At this time, the 3D scanner can generate a continuous set of data by dividing each part of the target object from various angles.

A process of forming a 3D image of a target object based on a set of continuous data generated by the 3D scanner may be referred to as registration.

The registration process may be a process of mapping two or more data sets into one coordinate system. A coordinate system may be assigned to each of the data obtained by scanning or otherwise. Some of the data may be allocated to a fixed system, and the remaining data may be allocated to a moving system. A moving system can be transformed in virtual space to converge on a fixed system. Such transformations may include rotational transformations or translational transformations.

The plurality of scan data for one object is data obtained by scanning different parts of the object, but each of the plurality of scan data may include overlapping parts. The matching process may match the overlapping portions of each of the plurality of scan data, and generate 3D scan data of the object based on the plurality of scan data.

As the 3D scanner technology develops, attempts to apply the 3D scanner to various processes are increasing.

In particular, since the use of a 3D scanner is essential in reverse engineering the body structure, the demand for convenient and accurate reverse engineering of the body structure using 3D scan data is increasing, and research on related technologies is required. .

Korean Patent Registration No. 10-2369093 Korean Patent No. 10-2237374 Korean Patent Registration No. 10-0933715 Korean Patent Publication No. 10-2020-0013416

According to an embodiment, when the plurality of parts are respectively scanned by the 3D scanner in a state in which the first vehicle is disassembled into a plurality of parts, the first 3D scan data that is generated by scanning the plurality of parts from the 3D scanner, respectively Acquire scan data, analyze which part is 3D scan data generated by scanning which part of the plurality of scan data included in the first scan data, and 1-1 scan data of the first scan data scans the first part If it is analyzed as the 3D scan data generated by An object of the present invention is to provide a method, apparatus, and system for reverse engineering a body structure using 3D scan data.

Objects of the present invention are not limited to the objects mentioned above, and other objects not mentioned will be clearly understood from the description below.

According to an embodiment, in a method for reverse-engineering a body structure using 3D scan data, performed by an apparatus, in a state in which a first vehicle is disassembled into a plurality of parts, the plurality of parts are configured by a 3D scanner obtaining first scan data, which is 3D scan data generated by scanning each of the plurality of parts from the 3D scanner, respectively; analyzing which part of the plurality of scan data included in the first scan data is 3D scan data generated by scanning each; If the 1-1 scan data among the first scan data is analyzed to be 3D scan data generated by scanning a first part, identifying a first point, which is a location where the first part is installed, in the first vehicle; ; and reverse-engineering the body structure around the first point based on the first-first scan data.

The method for reverse engineering a body structure using the 3D scan data may include: generating a first input signal by encoding the 1-1 scan data for each pixel; inputting the first input signal to an artificial neural network, and obtaining a first output signal as an output of the first input signal; generating a classification result for the surface of the first part for each pixel based on the first output signal; and setting a degree of wear on the surface of the first part for each pixel based on the classification result of the surface of the first part.

In the method for reverse engineering the body structure using the 3D scan data, the wear level on the surface of the first part is checked for each pixel, a region with a lower degree of wear than a preset reference value is divided into a normal area, and an area with a higher degree of wear than the reference value is identified. dividing the area into damaged areas; calculating a first ratio that is a ratio of an area occupied by the normal region on the surface of the first part, and calculating a second ratio that is a ratio of an area occupied by the damaged region on the surface of the first part; checking whether the first ratio is higher than a preset first reference ratio; classifying the first part as a part to be used for reassembly through reverse engineering when it is confirmed that the first ratio is higher than the first reference ratio; when it is confirmed that the first ratio is lower than the first reference ratio, checking whether the second ratio is lower than a preset second reference ratio; classifying the first part as a part requiring repair when it is confirmed that the second ratio is lower than the second reference ratio; and classifying the first part as a part requiring replacement when it is confirmed that the second ratio is higher than the second reference ratio.

According to an embodiment, in a method for reverse engineering a body structure using 3D scan data, when the plurality of parts are scanned by a 3D scanner in a state in which the first vehicle is disassembled into a plurality of parts, obtaining first scan data that is 3D scan data generated by scanning the plurality of parts from the 3D scanner, respectively; analyzing which part of the plurality of scan data included in the first scan data is 3D scan data generated by scanning each; If the 1-1 scan data among the first scan data is analyzed to be 3D scan data generated by scanning a first part, identifying a first point, which is a location where the first part is installed, in the first vehicle; ; and reverse-engineering the body structure around the first point based on the 1-1 scan data, so that the body structure can be conveniently and accurately reverse-engineered using the 3D scan data.

On the other hand, the effects according to the embodiments are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

1 is a diagram schematically showing the configuration of a system according to an embodiment.
2 is a flowchart illustrating a process of reverse engineering a vehicle body structure using 3D scan data according to an exemplary embodiment.
3 is a diagram illustrating 3D scan data according to an exemplary embodiment.
4 is a flowchart illustrating a process of setting a wear level on a surface of a component according to an exemplary embodiment.
5 is a view showing a result of setting the degree of wear on the surface of the component according to an embodiment.
6 is a flowchart illustrating a process of determining whether repair and replacement of parts is necessary according to an exemplary embodiment.
7 is a flowchart illustrating a process of estimating the size of a component according to an exemplary embodiment.
8 is a flowchart illustrating a process of recording and storing an image of a part according to an exemplary embodiment.
9 is a diagram for explaining an artificial neural network according to an embodiment.
10 is a diagram for explaining a method of learning an artificial neural network according to an embodiment.
11 is an exemplary diagram of a configuration of an apparatus according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes may be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It should be understood that all modifications, equivalents and substitutes for the embodiments are included in the scope of the rights.

Specific structural or functional descriptions of the embodiments are disclosed for purposes of illustration only, and may be changed and implemented in various forms. Accordingly, the embodiments are not limited to a specific disclosure form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Although terms such as first or second may be used to describe various elements, these terms should be interpreted only for the purpose of distinguishing one element from another. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected” to another component, it may be directly connected or connected to the other component, but it should be understood that another component may exist in between.

Terms used in the examples are used for the purpose of description only, and should not be construed as limiting. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present specification, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In addition, in the description with reference to the accompanying drawings, the same components are assigned the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted. In the description of the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description thereof will be omitted.

The embodiments may be implemented in various types of products, such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent cars, kiosks, wearable devices, and the like.

An artificial intelligence (AI) system is a computer system that implements human-level intelligence, and unlike the existing rule-based smart system, the machine learns and makes decisions on its own. The more the AI system is used, the better the recognition rate and the more accurate understanding of user preferences, and the existing rule-based smart systems are gradually being replaced by deep learning-based AI systems.

Artificial intelligence technology consists of machine learning and element technologies using machine learning. Machine learning is an algorithm technology that categorizes/learns characteristics of input data by itself, and element technology uses machine learning algorithms such as deep learning to simulate functions such as cognition and judgment of the human brain. It consists of technical fields such as understanding, reasoning/prediction, knowledge expression, and motion control.

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology for recognizing and applying/processing human language/text, and includes natural language processing, machine translation, dialogue system, question and answer, and speech recognition/synthesis. Visual understanding is a technology for recognizing and processing objects like human vision, and includes object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image improvement, and the like. Inferential prediction is a technology for logically reasoning and predicting by judging information, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendation. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data generation/classification) and knowledge management (data utilization). Motion control is a technology for controlling autonomous driving of a vehicle and movement of a robot, and includes motion control (navigation, collision, driving), manipulation control (action control), and the like.

In general, in order to apply the machine learning algorithm to real life, learning is performed in the Trial and Error method due to the characteristics of the basic methodology of machine learning. In particular, deep learning requires hundreds of thousands of iterations. It is impossible to execute this in the actual physical external environment, so instead, the actual physical external environment is implemented on a computer and learning is performed through simulation.

1 is a diagram schematically showing the configuration of a system according to an embodiment.

Referring to FIG. 1 , a system according to an embodiment may include a 3D scanner 100 and an apparatus 200 .

The 3D scanner 100 may generate 3D scan data by scanning the analysis target from various angles. Here, the 3D scan data is a 3D image generated by scanning an analysis target.

The 3D scanner 100 may include a communication module, and may be configured to communicate with the device 200 by wire or wireless.

The device 200 may be a self-server owned by a person or organization that provides services using the device 200, a cloud server, or a peer-to-peer (p2p) set of distributed nodes. may be The device 200 may be configured to perform all or part of an arithmetic function, a storage/referencing function, an input/output function, and a control function of a typical computer. The device 200 may include at least one artificial neural network that performs an inference function.

The device 200 may be configured to communicate with the 3D scanner 100 by wire or wireless, and may control the overall operation of the 3D scanner 100 .

When the device 200 obtains 3D scan data from the 3D scanner 100 , the device 200 may reverse engineer the body structure using the 3D scan data.

The device 200 may classify the surface of the vehicle part according to cracks based on artificial intelligence in order to set the degree of wear on the surface of the vehicle part, and for this, it may include a pre-trained artificial neural network.

In the present invention, artificial intelligence (AI) refers to a technology that imitates human learning ability, reasoning ability, perceptual ability, etc., and implements it with a computer, and the concepts of machine learning, symbolic logic, etc. may include Machine Learning (ML) is an algorithm technology that classifies or learns characteristics of input data by itself. Artificial intelligence technology is an algorithm of machine learning that can analyze input data, learn the results of the analysis, and make judgments or predictions based on the results of the learning. In addition, technologies that use machine learning algorithms to simulate functions such as cognition and judgment of the human brain can also be understood as a category of artificial intelligence. For example, technical fields such as verbal comprehension, visual comprehension, reasoning/prediction, knowledge expression, and motion control may be included.

Machine learning may refer to the process of training a neural network model using experience of processing data. With machine learning, computer software could mean improving its own data processing capabilities. The neural network model is constructed by modeling the correlation between data, and the correlation may be expressed by a plurality of parameters. A neural network model extracts and analyzes features from given data to derive correlations between data, and repeating this process to optimize parameters of a neural network model can be called machine learning. For example, a neural network model may learn a mapping (correlation) between an input and an output with respect to data given as an input/output pair. Alternatively, the neural network model may learn the relationship by deriving regularity between the given data even when only input data is given.

The artificial intelligence learning model or neural network model may be designed to implement a human brain structure on a computer, and may include a plurality of network nodes that simulate neurons of a human neural network and have weights. A plurality of network nodes may have a connection relationship with each other by simulating a synaptic activity of a neuron through which a neuron sends and receives a signal through a synapse. In the AI learning model, a plurality of network nodes can exchange data according to a convolutional connection relationship while being located in layers of different depths. The artificial intelligence learning model may be, for example, an artificial neural network model, a convolutional neural network model, or the like. As an embodiment, the AI learning model may be machine-learned according to a method such as supervised learning, unsupervised learning, reinforcement learning, or the like. Machine learning algorithms for performing machine learning include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, Ada-boost. , Perceptron, Genetic Programming, Clustering, etc. may be used.

Among them, CNNs are a type of multilayer perceptrons designed to use minimal preprocessing. CNN consists of one or several convolutional layers and general artificial neural network layers on top of it, and additionally utilizes weights and pooling layers. Thanks to this structure, CNN can fully utilize the input data of the two-dimensional structure. Compared with other deep learning structures, CNN shows good performance in both video and audio fields. CNNs can also be trained through standard back-passing. CNNs are easier to train than other feedforward artificial neural network techniques and have the advantage of using fewer parameters.

Convolutional networks are neural networks that contain sets of nodes with bound parameters. The increased size of available training data and the availability of computational power, combined with advances in algorithms such as piecewise linear units and dropout training, have greatly improved many computer vision tasks. For huge data sets, such as those available for many tasks today, overfitting is not important, and increasing the size of the network improves test accuracy. Optimal use of computing resources becomes a limiting factor. To this end, a distributed, scalable implementation of deep neural networks may be used.

FIG. 2 is a flowchart illustrating a process of reverse engineering a vehicle body structure using 3D scan data according to an exemplary embodiment, and FIG. 3 is a diagram illustrating 3D scan data according to an exemplary embodiment.

Referring to FIG. 2 , first, in step S201 , in the device 200 in a state in which the first vehicle is disassembled into a plurality of parts, when the plurality of parts are scanned by the 3D scanner 100 respectively, the 3D scanner 100 ) to obtain first scan data, which is 3D scan data generated by scanning a plurality of parts, respectively.

Specifically, in a state in which the first vehicle is disassembled into a plurality of parts, the 3D scanner 100 scans each of the plurality of parts to generate scan data for each of the plurality of parts, as shown in FIG. 3 . can In this case, the 3D scanner 100 may generate 3D scan data by scanning each of the plurality of parts, and may collect 3D scan data for each of the plurality of parts to generate the collected scan data as first scan data. have. Thereafter, the device 200 may acquire first scan data from the 3D scanner 100 .

For example, when the first vehicle is in a state of being disassembled into a first part, a second part, and a third part, the 3D scanner 100 may scan the first part to generate 1-1 scan data, , scan the second part to generate 1-2 scan data, scan the third part to generate 1-3 scan data, 1-1 scan data, 1-2 scan data And by collecting the 1-3 scan data, scan data including the 1-1 scan data, the 1-2 scan data, and the 1-3 scan data may be generated as the first scan data. Thereafter, the apparatus 200 may obtain first scan data including the 1-1 scan data, the 1-2 scan data, and the 1-3 scan data from the 3D scanner 100 .

Before acquiring the scan data from the 3D scanner 100 , the device 200 obtains identification information of the first vehicle from the 3D scanner 100 , or obtains identification information of the first vehicle from a terminal connected to the 3D scanner 100 . can be obtained Here, the identification information of the first vehicle is information capable of identifying the first vehicle, and may include the type and model name of the first vehicle.

After obtaining the identification information of the first vehicle, when the device 200 obtains the first scan data from the 3D scanner 100, the first scan data can be identified as scan data generated by scanning parts of the first vehicle. have.

In step S202 , the apparatus 200 may analyze which part of the plurality of scan data included in the first scan data is 3D scan data generated by scanning each part. At this time, the device 200 checks the shape of each part through each of the plurality of scan data included in the first scan data, and compares the shape of each part with the shape of the previously registered part to scan which part each part You can analyze what you did. To this end, 3D scan data registered for each part of the vehicle is stored in the database of the device 200 .

For example, when the first scan data includes the 1-1 scan data, the 1-2 scan data, and the 1-3 scan data, the device 200 stores the 1-1 scan data and the data stored in the database. By comparing the 3D scan data of the parts, if it is determined that the shape of the part identified through the 1-1 scan data matches or is most similar to the shape of the part identified through the 3D scan data of the first part, 1-1 The scan data may be analyzed as 3D scan data generated by scanning the first part. In addition, the apparatus 200 compares the 1-2 scan data with the 3D scan data of the parts stored in the database, and confirms the shape of the part identified through the 1-2 scan data and the 3D scan data of the second part When it is determined that the shapes of the parts are identical or most similar, the 1-2 scan data may be analyzed as 3D scan data generated by scanning the second part. In addition, the device 200 compares the 1-3 scan data with the 3D scan data of the parts stored in the database, and confirms the shape of the part identified through the 1-3 scan data and the 3D scan data of the third part When it is determined that the shapes of the parts are identical or most similar, the 1-3 scan data may be analyzed as 3D scan data generated by scanning the third part.

In operation S203 , the apparatus 200 may analyze 1-n-th scan data among the first scan data as 3D scan data generated by scanning the n-th part.

For example, the apparatus 200 may analyze the first-first scan data among the first scan data as 3D scan data generated by scanning the first part.

In step S204, the device 200 analyzes that the 1-n-th scan data among the first scan data is 3D scan data generated by scanning the n-th part, the n-th part being the position where the n-th part is installed in the first vehicle. n points can be identified. To this end, in the database of the device 200 , the location where each part is installed in the vehicle is stored as 3D coordinate information.

For example, if the device 200 analyzes that the 1-1 scan data among the first scan data is 3D scan data generated by scanning the first part, the device 200 determines the position where the first part is installed in the first vehicle. 1 point can be checked. In this case, the device 200 may search the database to obtain 3D coordinate information on the first part, and then check the first point by using the 3D coordinate information on the first part.

In step S205 , when the location where the n-th part is installed in the first vehicle is identified as the n-th point, the device 200 reverse-engineers the body structure around the first point based on the 1-n-th scan data. can do.

For example, when the location where the first part is installed in the first vehicle is confirmed as the first point, the device 200 reverse-engineers the body structure around the first point based on the 1-1 scan data. can do.

In step S206 , the device 200 may determine whether there is unanalyzed scan data among the first scan data.

If it is confirmed in step S206 that there is unanalyzed scan data, the process returns to step S203 and the processes from step S203 to step S206 may be performed again.

For example, when the body structure around the first point is reverse-engineered based on the 1-1 scan data, the device 200 may be set as having completed the analysis of the 1-1 scan data, , when it is confirmed that only the analysis of the 1-1 scan data among the first scan data is completed and the analysis of the 1-2 scan data is not completed, the 1-2 scan data is 3D generated by scanning the second part Analyze as scan data, check the second point where the second part is installed in the first vehicle, and reverse engineer the body structure around the second point based on the 1-2 scan data have.

When the body structure is reverse-engineered, when the first part and the second part are located adjacent to each other, the apparatus 200 determines the first part and the second part based on the 1-1 scan data and the 1-2 scan data. It is possible to reverse engineer the body structure for the connecting parts of the parts.

The method of reverse engineering the body structure based on the scan data may be performed by applying various known techniques.

4 is a flowchart for explaining a process of setting the degree of wear on the surface of the part according to an embodiment, and FIG. 5 is a view showing the result of setting the degree of wear on the surface of the part according to the embodiment.

Referring to FIG. 4 , first, in step S401 , the device 200 may generate a first input signal by encoding the first-first scan data for each pixel.

Specifically, the device 200 may generate a first input signal by identifying a 3D image through the 1-1 scan data and encoding pixels of the 3D image with color information. The color information may include, but is not limited to, RGB color information, brightness information, saturation information, and depth information. The device 200 may convert color information into a numerical value, and may encode the first-first scan data for each pixel in the form of a data sheet including the value.

In step S402 , the device 200 may input the first input signal to the pre-trained artificial neural network in the device 200 .

The artificial neural network according to an embodiment is composed of a feature extraction neural network and a classification neural network, and the feature extraction neural network sequentially stacks a convolutional layer and a pooling layer on an input signal. The convolution layer includes a convolution operation, a convolution filter, and an activation function. The calculation of the convolution filter is adjusted according to the matrix size of the target input, but a 9X9 matrix is generally used. The activation function generally uses, but is not limited to, a ReLU function, a sigmoid function, and a tanh function. The pooling layer is a layer that reduces the size of the input matrix, and uses a method of extracting representative values by tying pixels in a specific area. In general, the average value or the maximum value is often used for the calculation of the pooling layer, but is not limited thereto. The operation is performed using a square matrix, usually a 9x9 matrix. The convolutional layer and the pooling layer are repeated alternately until the corresponding input becomes small enough while maintaining the difference.

According to an embodiment, the classification neural network has a hidden layer and an output layer. Classification of artificial neural networks for classifying crack stages on the surface of parts The neural network consists of 5 or less hidden layers, and may include a total of 50 or less hidden layer nodes. The activation function of the hidden layer uses a ReLU function, a sigmoid function, and a tanh function, but is not limited thereto. There is a total of one output layer node of the classification neural network, and an output value for classification of the surface of a part can be output to the output layer node. A detailed description of the artificial neural network will be described later with reference to FIG. 9 .

In step S403 , the apparatus 200 may obtain a first output signal as an output of the first input signal based on a result of the input of the artificial neural network. Here, the first output signal may be generated to correspond to the first input signal. That is, when the first input signal is generated by encoding n pixels, the first output signal may include output values for the n pixels.

In operation S404 , the device 200 may generate a classification result for the surface of the first part for each pixel based on the first output signal. Here, the classification result may include information on which stage of cracking the surface of the part is classified into.

For example, when the output value of the first pixel is 1 as a result of checking the output value of the first output signal, the device 200 generates a classification result as that the surface of the first pixel corresponds to the crack stage 1, When the output value for 2 pixels is 2, the classification result may be generated as the surface of the second pixel corresponds to the second crack stage. The higher the crack stage, the more severe the cracks on the surface.

In operation S405 , the device 200 may set the wear level of the surface of the first part for each pixel based on the classification result of the surface of the first part.

For example, if it is determined that the surface of the first pixel on the surface of the first part corresponds to crack stage 1, the device 200 sets the wear level of the first pixel to 1, and the surface of the first pixel is cracked 2 If it is confirmed that the step corresponds to the first pixel, the wear level of the first pixel may be set to 2 .

When the wear level on the surface of the first part is set for each pixel, as shown in FIG. 5 , the wear degree on the surface of the first part may be displayed in different colors according to the set value.

6 is a flowchart illustrating a process of determining whether repair and replacement of parts is necessary according to an exemplary embodiment.

Referring to FIG. 6 , first, in step S601 , the device 200 may identify a wear level on the surface of the first component for each pixel, and distinguish a normal region from a damaged region. In this case, the apparatus 200 may classify an area having a lower degree of wear than the reference value as a normal area, and may classify an area having a higher degree of wear than the reference value as a damaged area. Here, the reference value may be set differently depending on the embodiment.

In step S602, the device 200 calculates a first ratio that is the ratio of the area occupied by the normal region on the surface of the first part, and calculates a second ratio that is the ratio of the area occupied by the damaged region on the surface of the first part. can

In step S603 , the device 200 may determine whether the first ratio is higher than the first reference ratio. Here, the first reference ratio may be set differently depending on the embodiment.

If it is determined in step S603 that the first ratio is higher than the first reference ratio, in step S605 , the apparatus 200 may classify the first part as a part to be used for reassembly through reverse engineering.

For example, when the first reference ratio is set to 90%, the apparatus 200 may classify the first part as a part to be used for reassembly through reverse engineering when it is confirmed that the first ratio is 95%.

If it is determined in step S603 that the first ratio is lower than the first reference ratio, in step S604 , the device 200 may determine whether the second ratio is lower than the second reference ratio. Here, the second reference ratio may be set differently according to embodiments.

If it is determined in step S604 that the second ratio is lower than the second reference ratio, in step S606 , the device 200 may classify the first part as a part requiring repair.

For example, when the second reference ratio is set to 20% and the second reference ratio is 15%, the device 200 may classify the first part as a part requiring repair.

If it is determined in step S604 that the second ratio is higher than the second reference ratio, in step S607 , the device 200 may classify the first part as a part requiring replacement.

For example, when the second reference ratio is set to 20%, the device 200 may classify the first part as a part requiring replacement when it is confirmed that the second ratio is 25%.

7 is a flowchart illustrating a process of estimating the size of a component according to an exemplary embodiment.

Referring to FIG. 7 , first, in step S701 , when the first vehicle is scanned by the 3D scanner 100 in a state in which the first vehicle is not disassembled, the device 200 receives the first from the 3D scanner 100 . Second scan data that is 3D scan data generated by scanning the vehicle may be acquired.

In step S702 , the device 200 may identify a plurality of parts installed in the first vehicle based on the second scan data.

Specifically, 3D scan data registered for each part of the first vehicle is stored in the database of the device 200, and the device 200 compares the second scan data with the 3D scan data of the parts stored in the database, 1 You can check each of a plurality of parts installed in a vehicle.

In step S703 , as a result of checking the plurality of parts installed in the first vehicle, the device 200 may determine that the first part and the second part are installed in the first vehicle.

If the device 200 determines that the first part and the second part are installed in the first vehicle, in step S704 , based on the second scan data, the first area that is the area occupied by the first part in the first vehicle , and in step S705 , a second area that is an area occupied by the second component in the first vehicle may be detected based on the second scan data.

In detecting the first area, the device 200 analyzes the characteristics of the shape of the first part in the first vehicle based on the second scan data, recognizes the boundary of the first part, and then recognizes the recognized second part. The first region may be detected through the boundary of one part. In this case, the apparatus 200 may detect the first region in the form of a two-dimensional box in a three-dimensional space.

In addition, in detecting the second region, the device 200 analyzes the characteristics of the shape of the second part in the first vehicle based on the second scan data, recognizes the boundary of the second part, and then recognizes The second region may be detected through the boundary of the second part. In this case, the apparatus 200 may detect the second region in the form of a two-dimensional box in a three-dimensional space.

In operation S706 , the apparatus 200 may estimate the length of the first area in the first vehicle based on the second scan data, and set the first length as the estimated length.

For example, when the first region is detected in the form of a two-dimensional box in the three-dimensional space, the apparatus 200 checks the lengths of the long axis and the diagonal of the first region, and uses the average value of the long axis and the diagonal of the first region. The first length may be set.

In operation S707 , the device 200 may estimate the length of the second area in the first vehicle based on the second scan data, and set the second length as the estimated length.

For example, when the second region is detected in the form of a two-dimensional box in the three-dimensional space, the device 200 checks the lengths of the long axis and the diagonal of the second region, and uses the average value of the long axis and the diagonal of the second region. A second length may be set.

In step S708 , the device 200 may identify a third length that is an actual length of the first part. To this end, the database of the device 200 stores information on the actual length divided by parts of the vehicle, and the device 200 retrieves the information stored in the database to determine the third length, which is the actual length of the first part. can be checked

In operation S709 , the apparatus 200 may estimate a fourth length that is an actual length of the second part by using the first length, the second length, and the third length.

Specifically, the apparatus 200 calculates a length ratio, which is a ratio of the first length and the third length, applies the second length to the calculated length ratio, and calculates the fourth length through the process of calculating the fourth length. can be estimated

For example, if the first length is 10 cm, the second length is 20 cm, and the third length is 15 cm, the device 200 may use the first length and the third length to calculate a length ratio of 1.5, , by applying the second length to the length ratio, the fourth length may be calculated as 30 cm.

8 is a flowchart illustrating a process of recording and storing an image of a part according to an exemplary embodiment.

Referring to FIG. 8 , first, in step S801 , when the first part is being photographed through the camera, the device 200 may acquire image information generated by the photographing of the first part from the camera. In this case, the camera is capturing the first part, and the device 200 may acquire image information generated by capturing the first part in real time. To this end, the camera may be configured to communicate with the device 200 in a wired or wireless manner.

In step S802 , the device 200 may extract the video image of the first viewpoint from the video information as the first image. That is, the device 200 may capture the image of the first viewpoint from the image information, and extract the captured image as the first image. Here, the first time point may be set differently depending on the embodiment.

In step S803 , the device 200 may classify an area occupied by the first part in the first image as the first area.

Specifically, the device 200 may analyze the first image to recognize the first part on the first image, and confirm the position of the first part located on the leftmost side on the first image as the first position. and confirming the position of the first part located on the uppermost side on the first image as the second position, confirming the position of the first part located on the rightmost side on the first image as the third position, and on the first image The position of the first part located at the lowermost position may be confirmed as the fourth position.

Then, the device 200 generates a first straight line extending in the vertical direction based on the first position, generates a second straight line extending in the left and right direction based on the second position, and vertically based on the third position. A third straight line extending in the direction may be generated, and a fourth straight line extending in a left and right direction based on the fourth position may be generated.

Thereafter, the device 200 may distinguish the first region from the first image after setting the region connecting the first straight line, the second straight line, the third straight line, and the fourth straight line as the first region.

In operation S804 , the device 200 may divide a portion having the first region in the first image, and extract the divided image as the 1-1 image.

In step S805 , the device 200 may check whether it is recognized that there is a foreign object in the 1-1 image. That is, the device 200 may check whether foreign substances in a form other than the first part are recognized on the 1-1 image, and may determine whether there is a foreign substance in the 1-1 image.

If it is determined in step S805 that there is a foreign material in the 1-1 image, in step S806 , the device 200 may classify an area occupied by the foreign material in the 1-1 image as a second area. In this case, the device 200 may distinguish the second region from the 1-1 image through the same method as the method for distinguishing the first region from the first image.

In step S807 , the device 200 may analyze the image information by reproducing the image information in reverse chronological order from the first time point.

In step S808 , as a result of analyzing and reproducing the image information in reverse chronological order from the first time point, the device 200 may confirm that there is no foreign material in the second area at the second time point.

In step S809 , the device 200 may extract the video image of the second viewpoint from the video information as the second image. That is, the device 200 may capture the image of the second viewpoint from the image information, and extract the captured image as the second image.

In operation S810 , the device 200 may extract a 2-1 image by dividing a portion having the second region from the second image.

In step S811 , the device 200 may replace the portion having the second region in the 1-1 image with the 2-1 image.

Specifically, when it is confirmed that there is a foreign material in the 1-1 image, the device 200 may classify the area occupied by the foreign material in the 1-1 image as a second area, and in the 1-1 image, the second area A 1-1 image in which the 1-1 image and the 2-1 image are combined may be generated by deleting the portion with the 2-1 image and inserting the 2-1 image at the deleted position. That is, the part with the second zone in the 1-1 image is replaced with the 2-1 image, the foreign material in the 1-1 image is deleted, and a part without the foreign material is added to the deleted position, -1 Foreign objects may no longer be recognized in the image.

9 is a diagram for explaining an artificial neural network according to an embodiment.

The artificial neural network 900 according to an embodiment may receive as an input an input signal generated by encoding scan data, and output information on which stage of crack the surface of the part is classified into.

Encoding according to an embodiment may be performed by storing color information for each pixel of an image in the form of a digitized data sheet, and the color information includes RGB color, brightness information, saturation information, and depth information of one pixel. can, but is not limited to.

According to an embodiment, the artificial neural network 900 is composed of a feature extraction neural network 910 and a classification neural network 920, and the feature extraction neural network 910 separates a part region and a background region from an image. In addition, the classification neural network 920 may perform an operation of determining whether the surface of the part is classified as a crack in the image.

The method in which the feature extraction neural network 910 distinguishes the part region and the background region is that the change in each value of color information from the data sheet of the input signal encoding the image is 30% in 6 or more of 8 pixels including one pixel. A bundle of pixels detected as having an abnormal change may be used as a boundary between the part area and the background area, but is not limited thereto.

The feature extraction neural network 910 stacks the input signal by sequentially stacking a convolutional layer and a pooling layer. The convolution layer includes a convolution operation, a convolution filter, and an activation function. The calculation of the convolution filter is adjusted according to the matrix size of the target input, but a 9X9 matrix is generally used. The activation function generally uses, but is not limited to, a ReLU function, a sigmoid function, and a tanh function. The pooling layer is a layer that reduces the size of the input matrix, and uses a method of extracting representative values by tying pixels in a specific area. In general, the average value or the maximum value is often used for the calculation of the pooling layer, but is not limited thereto. The operation is performed using a square matrix, usually a 9x9 matrix. The convolutional layer and the pooling layer are repeated alternately until the corresponding input becomes small enough while maintaining the difference.

The classification neural network 920 checks the surface of the part region separated from the background through the feature extraction neural network 910, and checks whether the surface state of the predefined crack is similar to the step-by-step surface state, so that the surface of the part region is the type of crack. It can be determined whether or not they are classified into stages. Information stored in the database can be used to compare the crack phase with the surface state.

The classification neural network 920 has a hidden layer and an output layer, and is composed of five or less hidden layers, including a total of 50 or less hidden layer nodes, and the activation function of the hidden layer is a ReLU function and a sigmoid function. and tanh functions, but is not limited thereto.

The classification neural network 920 may include only one output layer node in total.

The output of the classification neural network 920 is an output value for which stage of the crack the part surface is classified, and may indicate which stage of the crack it corresponds to. For example, when the output value is 1, it may indicate that the part surface corresponds to the crack stage 1, and when the output value is 2, it may indicate that the part surface corresponds to the crack stage 2.

The output of the classification neural network 920 may be output to n when the input signal is generated by encoding n pixels.

According to an embodiment, the artificial neural network 900 may learn by receiving the first learning signal generated by the corrected correct answer input by the user when the user discovers a problem in the output according to the artificial neural network 900 . The problem of output according to the artificial neural network 900 may mean a case in which output values classified into different stages of cracking are output on the surface of the part.

The first learning signal according to an embodiment is created based on the error between the correct answer and the output value, and in some cases, SGD using delta, a batch method, or a method following a backpropagation algorithm may be used. The artificial neural network 900 performs learning by modifying an existing weight according to the first learning signal, and may use momentum in some cases. A cost function can be used to calculate the error, and a cross entropy function can be used as the cost function. Hereinafter, the learning contents of the artificial neural network 900 will be described with reference to FIG. 10 .

10 is a diagram for explaining a method of learning an artificial neural network according to an embodiment.

According to an embodiment, the learning apparatus may train the artificial neural network 900 . The learning device may be a separate entity different from the device 200 , but is not limited thereto.

According to an embodiment, the artificial neural network 900 includes an input layer to which training samples are input and an output layer to output training outputs, and may be trained based on a difference between the training outputs and the first labels. Here, the first labels may be defined based on a representative image registered for each stage of the crack. The artificial neural network 900 is connected to a group of a plurality of nodes, and is defined by weights between the connected nodes and an activation function that activates the nodes.

The learning apparatus may train the artificial neural network 900 using a Gradient Decent (GD) technique or a Stochastic Gradient Descent (SGD) technique. The learning apparatus may use a loss function designed by the outputs and labels of the artificial neural network 900 .

The learning apparatus may calculate a training error using a predefined loss function. The loss function may be predefined with a label, an output, and a parameter as input variables, where the parameter may be set by weights in the artificial neural network 900 . For example, the loss function may be designed in a Mean Square Error (MSE) form, an entropy form, or the like, and various techniques or methods may be employed in an embodiment in which the loss function is designed.

The learning apparatus may find weights affecting the training error by using a backpropagation technique. Here, the weights are relationships between nodes in the artificial neural network 900 . The learning apparatus may use the SGD technique using labels and outputs to optimize the weights found through the backpropagation technique. For example, the learning apparatus may update the weights of the loss function defined based on the labels, outputs, and weights using the SGD technique.

According to an embodiment, the learning apparatus may acquire representative images 1001 of each labeled training crack stage from the database. The learning apparatus may obtain pre-labeled information on each of the crack stage representative images 1001, and the crack stage representative images 1001 may be labeled according to the pre-classified crack stage.

According to an embodiment, the learning apparatus may acquire 1000 labeled training crack stage representative images 1001, and based on the labeled training crack stage representative images 1001, the first training crack stage vectors ( 1002) can be created. Various methods may be employed for extracting the first training crack step vectors 1002 .

According to an embodiment, the learning apparatus may obtain the first training outputs 1003 by applying the first training crack step-by-step vectors 1002 to the artificial neural network 900 . The learning apparatus may train the artificial neural network 900 based on the first training outputs 1003 and the first labels 1004 . The learning apparatus may train the artificial neural network 900 by calculating training errors corresponding to the first training outputs 1003 and optimizing the connection relationship of nodes in the artificial neural network 900 to minimize the training errors. .

11 is an exemplary diagram of a configuration of an apparatus according to an embodiment.

The device 200 according to one embodiment includes a processor 201 and a memory 202 . The processor 201 may include at least one of the devices described above with reference to FIGS. 1 to 10 , or perform at least one method described above with reference to FIGS. 1 to 10 . A person or organization using the apparatus 200 may provide a service related to some or all of the methods described above with reference to FIGS. 1 to 10 .

The memory 202 may store information related to the above-described methods or a program in which the methods described below are implemented. Memory 202 may be volatile memory or non-volatile memory.

The processor 201 may execute a program and control the device 200 . The code of the program executed by the processor 201 may be stored in the memory 202 . The device 200 may be connected to an external device (eg, a personal computer or a network) through an input/output device (not shown), and may exchange data through wired/wireless communication.

The device 200 may be used to train an artificial neural network or to use a learned artificial neural network. Memory 202 may include a learning or learned artificial neural network. The processor 201 may learn or execute an artificial neural network algorithm stored in the memory 202 . The apparatus 200 for learning an artificial neural network and the apparatus 200 for using the learned artificial neural network may be the same or may be separate.

The embodiments described above may be implemented by a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the apparatus, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

As described above, although the embodiments have been described with reference to the limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (3)

A method for reverse engineering a body structure using 3D scan data, performed by an apparatus, comprising:
In a state in which the first vehicle is disassembled into a plurality of parts, when the plurality of parts are respectively scanned by the 3D scanner, the first scan data that is 3D scan data generated by scanning the plurality of parts from the 3D scanner obtaining;
analyzing which part of the plurality of scan data included in the first scan data is 3D scan data generated by scanning each;
When it is analyzed that the 1-1 scan data among the first scan data is 3D scan data generated by scanning a first part, information on the first part is obtained from a database to obtain the first part in the first vehicle. confirming a first point, which is an installation location;
reverse-engineering a body structure around the first point based on the 1-1 scan data;
When the first vehicle is scanned by the 3D scanner in a state in which the first vehicle is not disassembled, obtaining second scan data that is 3D scan data generated by scanning the first vehicle from the 3D scanner ;
identifying a plurality of parts installed in the first vehicle based on the second scan data;
When it is confirmed that the first part and the second part are installed in the first vehicle, a first area that is an area occupied by the first part in the first vehicle is detected based on the second scan data, detecting a second area that is an area occupied by the second component in the first vehicle;
Based on the second scan data, by estimating the length of the first region in the first vehicle, the first length is set, and by estimating the length of the second region in the first vehicle, the second length is determined setting;
checking a third length that is an actual length of the first part using information stored in the database; and
estimating a fourth length that is an actual length of the second part by using the first length, the second length and the third length;
A body structure reverse engineering method using 3D scan data. The method of claim 1,
generating a first input signal by encoding the 1-1 scan data for each pixel;
inputting the first input signal to an artificial neural network, and obtaining a first output signal as an output of the first input signal;
generating a classification result for the surface of the first part for each pixel based on the first output signal; and
Based on the classification result for the surface of the first part, further comprising the step of setting the degree of wear on the surface of the first part for each pixel,
A body structure reverse engineering method using 3D scan data. 3. The method of claim 2,
checking the degree of wear on the surface of the first part for each pixel, classifying an area with a lower degree of wear than a preset reference value as a normal area, and classifying an area with a higher degree of wear than the reference value as a damaged area;
calculating a first ratio that is a ratio of an area occupied by the normal region on the surface of the first part, and calculating a second ratio that is a ratio of an area occupied by the damaged region on the surface of the first part;
checking whether the first ratio is higher than a preset first reference ratio;
classifying the first part as a part to be used for reassembly through reverse engineering when it is confirmed that the first ratio is higher than the first reference ratio;
when it is confirmed that the first ratio is lower than the first reference ratio, checking whether the second ratio is lower than a preset second reference ratio;
classifying the first part as a part requiring repair when it is confirmed that the second ratio is lower than the second reference ratio; and
When it is confirmed that the second ratio is higher than the second reference ratio, further comprising the step of classifying the first part as a part requiring replacement,
A body structure reverse engineering method using 3D scan data.

Images