CN114565834A

CN114565834A - Detection method, workbench, detection system and computer readable storage medium for power assembly bearing installation quality

Info

Publication number: CN114565834A
Application number: CN202111629082.2A
Authority: CN
Inventors: 李献; 黄立新; 张孟浩; 曹诚
Original assignee: SAIC Volkswagen Automotive Co Ltd
Current assignee: SAIC Volkswagen Automotive Co Ltd
Priority date: 2021-12-28
Filing date: 2021-12-28
Publication date: 2022-05-31

Abstract

The invention relates to a detection method, a workbench, a detection system and a computer readable storage medium for the bearing installation quality of a power assembly. The detection method comprises S1, acquiring image data of the powertrain support; s2, performing data processing on the image data; s3, respectively extracting the features of the image feature data of the first bolt and the image feature data of the second bolt through a convolution network; and S4, judging the output result of the convolution network through the full-connection network, and obtaining the mounting quality of the power assembly support. The invention provides a detection method, a workbench, a detection system and a computer readable storage medium for the installation quality of a power assembly support, which can automatically judge the installation quality based on an on-site installation image so as to improve the installation quality of the power assembly support.

Description

Detection method, workbench, detection system and computer readable storage medium for power assembly bearing installation quality

Technical Field

The invention relates to the technical field of vehicle detection, in particular to a detection method, a workbench, a detection system and a computer readable storage medium for the bearing and mounting quality of a power assembly.

Background

The mounting mass of the powertrain has great influence on the NVH (Noise, Vibration, Harshness/Noise, Vibration and Harshness) performance of the whole vehicle, such as start-stop shaking/idle shaking/engine running Noise and the like. By detecting the supporting and mounting quality of the power assembly of the production line and carrying out statistical analysis on the detection result, the reliability and stability of the offline quality of the product and the assembly quality of the production line can be continuously improved, and the production process is optimized if necessary.

FIG. 1 shows a schematic diagram of a powertrain support mounting mass test criteria. As shown, the engine mount 101 is mounted to the engine 104 by a first bolt 102 and a second bolt 103. Distances from the edges of the first bolt 102 and the second bolt 103 to the edge of the support mounting surface 105 are d1 and d2, respectively, and when d1 is d2, it is judged that the mounting quality of the powertrain support is good, and when d1 is d2, it is judged that the mounting of the powertrain support is skewed, and the mass of the entire vehicle is affected.

In the prior art, the installation quality of the power assembly support is checked by means of sampling on the spot of an experienced engineer in a workshop or manually sampling in a subsequent quality assurance workshop, a corresponding automatic detection method is not available, the efficiency is low, the cost of manpower and material resources is high, and particularly, effective inheritable experience cannot be formed for companies with factories in different regions. Because manual spot check is adopted, each vehicle of each production line cannot be checked, and real-time rework correction cannot be carried out, so that the offline quality of the power assembly support is unstable.

In the prior art, many quality inspection methods based on visual technology, such as flaw detection, part presence/absence detection, damage degree detection and the like, are available, most of these methods adopt a pre-trained convolutional network to perform migration learning of picture classification, the features extracted by the network for classification are global features based on the whole picture, and on the premise of big data, the training data is usually more than ten thousand levels, under this condition, the application fields (flaw/part presence/damage degree and the like) with obvious classification features can obtain good effects. However, the judgment of the good and bad installation quality of the powertrain support is based on only a small part of local areas in the pictures, and the picture classification task of the type is called fine-grained classification, namely the classification does not depend on the global features of the images, but only depends on the local features of the images, such as the classification of vehicles of different brands, the classification of birds of different varieties and the like, and is classified through the local features of the images. The difficulty of this type of task is much greater than that of the general (non-fine-grained classification) image classification, mainly: 1/category differences are small (vehicles of different brands look similar); 2/within-class variation is large (vehicles of the same brand may have relatively large variation on the contrary); 3/features that determine the classification result exist only in a local area (the position of the emblem in the vehicle classification) in the picture.

Disclosure of Invention

In view of the above problems in the prior art, the present invention provides a method for detecting the mounting quality of a power assembly support, a workbench, a detection system, and a computer readable storage medium, which can automatically determine the mounting quality based on a field mounting image, thereby improving the mounting quality of the power assembly support.

Specifically, the invention provides a detection method for the mounting quality of a power assembly support, wherein the mounting quality is based on a first bolt and a second bolt for mounting an engine support on an engine, and the detection method comprises the following steps:

s1, acquiring image data supported by the power assembly;

s2, performing data processing on the image data, comprising the steps of:

s21, performing data enhancement on the image data;

s22, carrying out target detection on the image data after data enhancement, and intercepting the image data of the connecting point position of the engine support and the engine; meanwhile, carrying out gesture recognition on the image data subjected to data enhancement;

s23, according to the result of gesture recognition, unifying the gesture of the intercepted image data of the connecting point position;

s24, capturing the image characteristic data of the first bolt and the image characteristic data of the second bolt from the image data with unified postures;

s3, respectively extracting the features of the image feature data of the first bolt and the image feature data of the second bolt through a convolution network;

and S4, judging the output result of the convolution network through a full-connection network, and obtaining the mounting mass of the power assembly support.

According to an embodiment of the invention, in step S1, it is determined whether the powertrain support is mounted by the station sensor, and if so, the powertrain support is photographed by the camera.

According to one embodiment of the invention, in step S21, the data enhancement includes random truncation, random rotation, random flipping, and color adjustment.

According to an embodiment of the present invention, in step S22, the model used for target detection is obtained based on migration learning of YOLO V5.

According to an embodiment of the present invention, in step S22, the gesture recognition performs a classification process on the image data subjected to data enhancement according to a shooting angle.

According to an embodiment of the present invention, before performing step S3, the intercepted image characteristic data of the first bolt and the intercepted image characteristic data of the second bolt are subjected to secondary data enhancement.

According to an embodiment of the present invention, in step S3, feature extraction is performed on the intercepted image feature data of the first bolt and the intercepted image feature data of the second bolt by using the identical convolution networks.

The invention also provides a workbench for supporting and installing the mass of the power assembly, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the steps of the detection method provided by the invention.

The invention also provides a detection system for the mounting quality of the power assembly support, which comprises a station sensor, a camera, a terminal computer and the workbench, wherein the station sensor, the camera and the terminal computer are in two-way communication with the workbench, the station sensor is used for judging whether the power assembly support is mounted, if the power assembly support is mounted, a finishing instruction is sent to the workbench, the workbench sends a shooting request to the camera according to the finishing instruction of the station sensor, the camera shoots the power assembly support according to the shooting request and transmits a shot image to the workbench, and the terminal computer can obtain a detection result stored by the workbench.

The invention also provides a computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the detection method provided by the invention.

The detection method, the workbench, the detection system and the computer readable storage medium for the power assembly support installation quality can automatically judge the installation quality based on the field installation image, and further improve the installation quality of the power assembly support.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

Drawings

The accompanying drawings, which are included to provide a further explanation of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention. In the drawings:

FIG. 1 shows a schematic diagram of a powertrain support mounting mass test criteria.

FIG. 2 is a block flow diagram illustrating a method of detecting a quality of a powertrain support mount in accordance with one embodiment of the present invention.

FIG. 3 shows a schematic diagram of data processing according to an embodiment of the invention.

FIG. 4 is a schematic diagram of a powertrain support mounting mass detection system according to one embodiment of the present invention.

Detailed Description

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.

The technical solution in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the application, its application, or uses. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.

It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.

The relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present application unless specifically stated otherwise. Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description. Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate. In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

It should be noted that the terms "first", "second", and the like are used to define the components, and are only used for convenience of distinguishing the corresponding components, and the terms have no special meanings unless otherwise stated, and therefore, the scope of protection of the present application is not to be construed as being limited. Further, although the terms used in the present application are selected from publicly known and used terms, some of the terms mentioned in the specification of the present application may be selected by the applicant at his or her discretion, the detailed meanings of which are described in relevant parts of the description herein. Further, it is required that the present application is understood not only by the actual terms used but also by the meaning of each term lying within.

In the process of researching methods for solving the detection task of the bearing installation quality of the power assembly, the existing quality inspection methods based on the vision technology are tried, but the ideal effect (the accuracy rate is below 70%) is not achieved. The main reasons for this are as follows:

● the classified features are determined by two local features of the image, and the two local features need to be compared to obtain a classification result (different from the previous image fine-grained classification task, usually a single local feature can determine the classification result);

● has a limited number of samples and a small total number of image samples can be used.

FIG. 2 is a block flow diagram illustrating a method of detecting a quality of a powertrain support mount in accordance with one embodiment of the present invention. As shown in the figure, the invention provides a detection method for the quality of the support mounting of the power assembly. The mounting mass is based on the engine supporting a first bolt and a second bolt mounted to the engine. The detection method comprises the following steps:

s1, acquiring image data of the powertrain support;

and S2, processing the image data to solve the problems of large data noise and unobvious features. The data processing comprises the following steps:

s21, performing data enhancement on the image data;

s22, carrying out target detection on the image data after data enhancement, and intercepting the image data of the connecting point position of the engine support and the engine; meanwhile, carrying out gesture recognition on the image data after data enhancement;

s23, according to the result of gesture recognition, unifying the gesture of the captured image data of the connecting point position;

s24, capturing image characteristic data of the first bolt and image characteristic data of the second bolt from the image data with unified postures;

and S4, judging the output result of the convolution network through the full-connection network, and obtaining the mounting quality of the power assembly support.

Preferably, in step S1, it is determined whether the powertrain support is mounted by the station sensor, and if so, the powertrain support is photographed by the camera to acquire image data.

Preferably, in step S21, the data enhancement includes random truncation, random rotation, random flipping, and color adjustment. The data enhancement process is only used during model training, and data enhancement is not performed during subsequent reasoning.

Preferably, in step S22, the model used for target detection is obtained based on migration learning of YOLO V5. The purpose of target detection is to intercept the location of the engine mount connection point to the engine.

Preferably, in step S22, the gesture recognition performs a classification process on the data-enhanced image data according to the shooting angle. The gesture recognition process is to avoid the influence caused by different shooting angles, for example, the situation that the artificial photograph may be rotated by 90 °/180 °/270 °, and the process is actually a classification network, and a Resnet18 network is used in the present embodiment in consideration of efficiency and classification difficulty. Through the classification processing of the shooting angles, the target interception data of the gesture recognition are all input into the subsequent processing process in the same state.

FIG. 3 shows a schematic diagram of data processing according to an embodiment of the invention. As shown, black arrows represent image data acquired in time series. The first image a is the image data of the engine mount and the engine at the connection point, which is obtained by performing the target detection on the enhanced image data. The second image B is image data obtained by unifying the postures of the captured image data at the connection point positions according to the result of the posture recognition. Since the positions of the first bolt and the second bolt are relatively fixed, the image feature data of the first bolt and the image feature data of the second bolt shown in the third diagram C1 and the fourth diagram C2, respectively, are obtained by simple cutting.

It is easy to understand that the reason why the local feature data of the first bolt and the second bolt are not directly intercepted from the original image data by the target detection method in the data processing process is as follows: the two local characteristics are the first bolt and the second bolt, and because the engine compartment has a lot of parts, a plurality of bolts similar to the first bolt and the second bolt appear in the picture, so that the model cannot judge which two bolts are needed; the reason why two local feature data are obtained by performing the target detection on the target intercepted data again is not taken: the target detection algorithm is much more complex than the basic image classification. Therefore, the local characteristic data of the first bolt and the second bolt are acquired by adopting the unification of the target detection and the gesture recognition modes, so that the whole data processing process is simpler and more efficient.

Preferably, before performing step S3, the image characteristic data of the first bolt and the image characteristic data of the second bolt are subjected to secondary data enhancement. Secondary data enhancement can effectively solve the problem of model overfitting, and the capability of extracting the characteristics of the convolution network is greatly improved. The secondary data enhancement adopts the conventional methods of random interception, random rotation, random inversion, color adjustment and the like, and also adopts the methods of adding salt and pepper noise and random erasure, and is proved to be really effective through experiments.

Preferably, in step S3, the extracted image feature data of the first bolt and the extracted image feature data of the second bolt are subjected to feature extraction by using the same convolution network, so as to obtain the same feature on different local feature data. The step S3 functions to perform feature extraction on the input image data to obtain a feature map. Since the input at this time is the local detail of the bolt extracted from the original image, the image features are less, and in order to avoid overfitting, a network with a smaller scale is selected, in the embodiment, a Resnet18 network is adopted, the last pooling layer and the full link layer are removed, and only the feature map which is output last by the network is used. Experiments have verified that better results cannot be obtained by using the Resnet34 and the Resnet50 networks, probably because more than one hundred live photographs and more than three hundred laboratory photographs are provided for training due to less data, and the final result is rather poor due to the over-fitting phenomenon of a complex model. It should be noted that the convolution networks used for the two local feature images are identical, that is, the image feature data of the first bolt and the image feature data of the second bolt are extracted by identical feature extractors (i.e., convolution networks), so as to extract the features of the first bolt and the second bolt from the two local feature images respectively. And finally, merging the two feature maps obtained by convolution and inputting the merged feature maps to a subsequent fully-connected network.

The invention also provides a workbench for supporting and installing the quality of the power assembly. The workstation includes a memory, a processor, and a computer program stored on the memory and executable on the processor. The steps of the detection method described above are implemented when the processor executes the computer program.

FIG. 4 is a schematic diagram of a powertrain support mounting mass detection system according to one embodiment of the present invention. As shown, the present invention also provides a powertrain support mounting mass detection system 400. The system comprises a station sensor 401, a camera 402, a terminal computer 403 and the workbench 404. Workstation sensor 401, camera 402 and terminal computer 403 are in two-way communication with workstation 404.

The station sensor 401 is configured to determine whether the power assembly support is completely installed, and if the power assembly support is completely installed, send a completion instruction to the workbench 404, the workbench 404 sends a shooting request to the camera 402 according to the completion instruction of the station sensor 401, the camera 402 shoots the power assembly support according to the shooting request, and transmits a shot image to the workbench 404, and the terminal computer 403 can obtain a detection result stored in the workbench 404. The pictures taken by the camera 402 and the corresponding quality detection results are stored in a database of the workstation 404. Engineers can check through the terminal computer 403 according to requirements, the terminal computer 403 can obtain a single detection result through the workbench 404, and statistical results of qualification rates of different factories/different vehicle types can be output according to time span.

The present invention also provides a computer readable storage medium having a computer program stored thereon. Which computer program, when being executed by a processor, carries out the steps of the detection method as described above.

The specific implementation and technical effects of the workbench and the computer-readable storage medium can be found in the above embodiments of the detection method provided by the present invention, and are not described herein again.

Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.

In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a web site, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk (disk) and disc (disc), as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk and blu-ray disc where disks (disks) usually reproduce data magnetically, while discs (discs) reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

The detection method, the workbench, the detection system and the computer readable storage medium for the power assembly supporting and mounting quality can automatically judge the mounting quality based on the field mounting image through a neural network method, verify that the final accuracy rate reaches more than 90% according to an actual test, further reduce the inspection cost, improve the product quality and improve the working efficiency. Has the following advantages:

1. and processing the input data by combining target detection and gesture recognition, and intercepting the characteristic data of the first bolt and the second bolt from the original input image data to serve as new input to a subsequent convolution network. This operation greatly reduces the noise of the image data, making the desired features easier to capture; because the final judgment result of the model is obtained by comparing the two local features, the original image is intercepted into the local feature images of the first bolt and the second bolt, so that the respective features of the two local sub-images can be respectively obtained, and the judgment result obtained by comparing the two local features becomes possible.

2. And respectively inputting the local feature data of the first bolt and the second bolt into the convolution network to obtain respective feature maps, and then merging the feature maps for classification judgment. The method of merging first is not adopted, because the classification result can be obtained only by comparing the features of the two local images, the classification result can be respectively input into a network to obtain the respective features of the two local images, the features are extracted by convolution after merging, the total features of the merged image are obtained, the respective features of the two local images cannot be obtained, and a correct result cannot be obtained by comparing the features.

It will be apparent to those skilled in the art that various modifications and variations can be made to the above-described exemplary embodiments of the present invention without departing from the spirit and scope of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.

Claims

1. A method of detecting a mount mass of a powertrain mount, the mount mass based on a first bolt and a second bolt of an engine mount to an engine, the method comprising the steps of:

s1, acquiring image data supported by the power assembly;

s2, performing data processing on the image data, including the steps of:

s21, performing data enhancement on the image data;

and S4, judging the output result of the convolution network through a full-connection network, and obtaining the mounting quality of the power assembly support.

2. The inspection method according to claim 1, wherein in step S1, it is determined whether the powertrain support is mounted by a station sensor, and if the powertrain support is mounted, the powertrain support is photographed by a camera.

3. The detection method according to claim 1, wherein in step S21, the data enhancement includes random truncation, random rotation, random flipping, and color adjustment.

4. The detection method according to claim 1, wherein in step S22, the model adopted for target detection is obtained based on migration learning of YOLO V5.

5. The detection method according to claim 1, wherein in step S22, the pose recognition performs a classification process on the image data subjected to data enhancement according to a photographing angle.

6. The inspection method according to claim 1, wherein the intercepted image feature data of the first bolt and the intercepted image feature data of the second bolt are subjected to secondary data enhancement before step S3 is executed.

7. The detection method according to claim 1, wherein in step S3, the identical convolution networks are used for feature extraction on the intercepted image feature data of the first bolt and the intercepted image feature data of the second bolt.

8. A power assembly support mounting mass table comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the computer program performs the steps of the method of testing as claimed in any one of claims 1 to 7.

9. A detection system for the mounting quality of a power assembly support is characterized by comprising a station sensor, a camera, a terminal computer and a workbench according to claim 8, wherein the station sensor, the camera and the terminal computer are in two-way communication with the workbench, the station sensor is used for judging whether the power assembly support is mounted or not, if the power assembly support is mounted, a completion instruction is sent to the workbench, the workbench sends a shooting request to the camera according to the completion instruction of the station sensor, the camera shoots the power assembly support according to the shooting request and transmits a shot image to the workbench, and the terminal computer can obtain a detection result stored in the workbench.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the detection method according to any one of claims 1 to 7.