CN110738644A - automobile coating surface defect detection method and system based on deep learning - Google Patents

automobile coating surface defect detection method and system based on deep learning Download PDF

Info

Publication number
CN110738644A
CN110738644A CN201910957565.1A CN201910957565A CN110738644A CN 110738644 A CN110738644 A CN 110738644A CN 201910957565 A CN201910957565 A CN 201910957565A CN 110738644 A CN110738644 A CN 110738644A
Authority
CN
China
Prior art keywords
image
deep learning
training
small
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910957565.1A
Other languages
Chinese (zh)
Inventor
常飞
董明宇
刘民
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CN201910957565.1A priority Critical patent/CN110738644A/en
Publication of CN110738644A publication Critical patent/CN110738644A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Quality & Reliability (AREA)
  • Artificial Intelligence (AREA)
  • Signal Processing (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the invention provides automobile coating surface defect detection methods and systems based on deep learning, which comprise the steps of obtaining detection images of areas to be detected on the surface of automobile coating, uniformly dividing the detection images into a plurality of small images with set sizes to form a small image test set, inputting the small image test set into a deep learning network model to obtain a detection result list corresponding to the small images with the set sizes, wherein the set sizes are N, N is more than or equal to 2, and performing de-duplication integration processing on the N detection result lists to obtain defect detection results.

Description

automobile coating surface defect detection method and system based on deep learning
Technical Field
The invention relates to the technical field of machine vision and image processing, in particular to automobile coating surface defect detection methods and systems based on deep learning.
Background
At present, the quality of an automobile coating surface in an automobile coating workshop is roughly sampled and evaluated in a manual visual inspection mode, namely, a specific area of the outer surface of an automobile body after coating and drying is detected, a detector visually searches for defects and marks the defects, and then manual statistical recording is carried out. After coating, the surface of the automobile body is smooth, the defects are small and weak, so that the defects are poor in visibility, and the defects can be effectively found only by frequently changing the visual direction and the observation distance. Therefore, the method of manual sampling and visual inspection has high omission ratio, low efficiency and long time consumption, and easily causes visual fatigue of detection personnel, thereby causing high error and uncertainty of manual detection data.
In addition, for detecting the surface defects of the automobile body, the existing detection device is usually arranged in a special dark field detection chamber, a plurality of linear array CCD cameras are adopted to scan two sides of the surface of the automobile body, and then the defects are detected by the traditional image processing and machine learning methods. The detection device has the advantages that the hardware cost is high, the occupied space is large, the detection area is limited in the gentle surface areas on the two sides of the automobile, the recall rate of the traditional image processing and machine learning algorithm to the small defect detection is low, the detection device is usually used for carrying out online rough evaluation on the surface defects of the automobile, the construction is required when a coating production line is constructed, and the detection device is not suitable for carrying out accurate quality evaluation on the surface quality of the automobile on the coating production line which completes the construction.
Based on the defects of the existing methods and devices for detecting the automobile coating surface of the automobile body, methods and systems for detecting the defects of the automobile coating surface, which are efficient, accurate and movable, are urgently needed.
Disclosure of Invention
The embodiment of the invention provides automobile coating surface defect detection methods and systems based on deep learning, which are used for solving the defects of low recognition rate, large error of an inspection algorithm, high hardware cost and large occupied space of automobile coating surface defect detection in the prior art.
, the embodiment of the invention provides deep learning-based automobile coating surface defect detection methods, which include:
acquiring a detection image of a to-be-detected area on the coating surface of the automobile; uniformly dividing a detection image into a plurality of small images with set sizes to build a small image test set; inputting the small image test set into a corresponding deep learning network model, and acquiring a detection result list corresponding to the small image with a set size; wherein the set sizes are N, and N is more than or equal to 2; processing the N detection result lists to obtain defect detection results;
the deep learning network model is obtained by training on the basis of small graph training samples with the set size of every and corresponding detection result labels.
, in the method for detecting defects on the painted surface of an automobile based on deep learning provided by the embodiment, each deep learning network model is established based on the YOLOv3 algorithm.
, the processing the N lists of test results includes de-dually integrating the N lists of test results based on a non-maximization suppression algorithm.
, before acquiring the inspection image of the area to be inspected on the painted surface of the automobile, adjusting the shooting distance and the illumination intensity of the area to be inspected.
, before inputting the small image test set to the corresponding deep learning network model, the method further includes a process of pre-training the deep learning network model, specifically:
the method comprises the steps of dividing a pre-training image of the automobile painting surface to obtain th pre-training small images with preset sizes, sequentially taking each pre-training small image as the input of a deep learning network model to realize iterative training of the deep learning network model, and obtaining the deep learning network model corresponding to th pre-training small images with preset sizes.
, the segmenting the pre-training image of the painted surface of the automobile includes computing a segmentation grid of the pre-training image, wherein the segmentation grid has a column number Gc＝floor(Lw/M), the number of lines of the split grid Gr＝floor(Lh(ii)/M); segmenting the pre-training image by utilizing a segmentation grid to obtain a plurality of pre-training small images; calculating the actual size of the pre-trained small image, wherein the width P of the pre-trained small imagew＝Lw/GcHeight P of the pre-trained small imageh＝Lh/Gr(ii) a Wherein floor (x) is an integer function, LWFor the width of the pre-training image, LhM is the th preset size for the height of the pre-training image.
The step , before the processing of the N detection result lists, further includes converting coordinates of upper left point and lower right point of the small images in each detection result list into coordinates of upper left point and lower right point of the detection images.
In a second aspect, an embodiment of the present invention provides deep learning-based automobile painting surface defect detection systems, including a visual inspection device and an image processing unit;
the image processing unit comprises an th processing subunit, a second processing subunit and a third processing subunit;
the processing subunit is used for dividing the detection image into a plurality of small images with set sizes uniformly to build a small image test set;
the second processing subunit is stored with a deep learning network model, and is used for inputting the small image test set into the deep learning network model and acquiring a detection result list corresponding to the small images with set sizes, wherein the set sizes are N, and N is more than or equal to 2, and the deep learning network model is obtained by training on the basis of small image training samples with set sizes of every and corresponding detection result labels;
and the third processing subunit is used for performing de-duplication integration processing on the N detection result lists to acquire a defect detection result.
, the visual inspection device comprises a wheel type chassis metal bracket and an image acquisition unit, wherein the image acquisition unit is fixedly arranged on the wheel type chassis metal bracket, and the wheel type chassis metal bracket is used for adjusting the height and the pitch angle of the image acquisition unit;
the image acquisition unit includes: the system comprises a laser range finder, an area array CCD camera, an LED light source and a light source brightness controller; the laser range finder is used for acquiring the distance from the area array CCD camera to the area to be detected on the automobile coating surface; the light source brightness controller is used for adjusting the brightness of the LED light source according to the distance.
In a third aspect, the embodiment of the invention provides automobile painting surface defect detection devices based on deep learning, which comprise a visual detection device and an image processing unit;
the visual detection device comprises a wheel type chassis metal support and an image acquisition unit;
the image acquisition unit is fixedly arranged on the wheel type chassis metal support, and the wheel type chassis metal support is used for adjusting the height and the pitch angle of the image acquisition unit; the image acquisition unit includes: the system comprises a laser range finder, an area array CCD camera, an LED light source and a light source brightness controller; the laser range finder is used for acquiring the distance from the area array CCD camera to the area to be detected on the automobile coating surface; the light source brightness controller is used for adjusting the brightness of the LED light source according to the distance;
the system comprises a vision processing device, an image processing unit, a deep learning network model and a defect detection result processing unit, wherein the vision processing device is used for obtaining a detection image of a to-be-detected area on the coating surface of an automobile, the image processing unit is used for uniformly dividing the detection image into a plurality of small images with set sizes to build a small image test set, the small image test set is input into the deep learning network model to obtain a detection result list corresponding to the small images with the set sizes, the set sizes are N, N is not less than 2, the N detection result lists are subjected to de-duplication integration processing to obtain the defect detection result, and the deep learning network model is obtained by training on the basis of small image training samples with the set sizes of and corresponding detection result labels.
In a fourth aspect, an embodiment of the present invention provides electronic devices, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the steps of the method for detecting defects on painted surfaces of automobiles based on deep learning as described in any of the .
In a fifth aspect, the present invention further provides computer readable storage media, wherein the computer program is used for implementing the steps of the method for detecting defects on the surface of painted surface of an automobile based on deep learning as described in any of the above when the computer program is executed by a processor.
According to the method and the system for detecting the defects of the automobile coating surface based on the deep learning, provided by the embodiment of the invention, the same detection image is divided by using different set sizes, small image test sets with different sizes are obtained and are respectively input into a pre-trained deep learning network model, so that a plurality of detection result lists aiming at the same detection image are obtained, and the defect detection result information contained in the detection image is obtained after different detection result lists are subjected to integrated processing.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, is briefly introduced in the drawings required in the description of the embodiments or the prior art, it is obvious that the drawings in the following description are embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic flow chart of methods for detecting surface defects based on automobile painting according to embodiments of the present invention;
FIG. 2 is a schematic flow chart of another methods for detecting surface defects based on automobile painting according to embodiments of the present invention;
FIG. 3 is a schematic structural diagram of types of systems for detecting surface defects based on automobile painting according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a visual inspection device of types of devices for detecting surface defects based on automobile painting according to an embodiment of the present invention;
fig. 5 is a schematic physical structure diagram of electronic devices according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete description of the technical solutions of the embodiments of the present invention will be given below with reference to the accompanying drawings of the embodiments of the present invention, and it is obvious that the described embodiments are partial embodiments of of the present invention, rather than all embodiments.
As shown in fig. 1, an embodiment of the present invention provides deep learning-based automobile painting surface defect detection methods, including but not limited to the following steps:
step S1, acquiring a detection image of a to-be-detected area on the surface of the automobile coating;
step S2, uniformly dividing the detection image into a plurality of small images with set sizes to build a small image test set;
step S3, inputting the small image test set into the deep learning network model, and acquiring a detection result list corresponding to the small image with a set size; wherein the set sizes are N, and N is more than or equal to 2;
step S4, carrying out de-duplication integration processing on the N detection result lists to obtain defect detection results;
the deep learning network model is obtained by training on the basis of small graph training samples with the set size of every and corresponding detection result labels.
In the prior art, when the surface defect detection of automobile coating is carried out on automobiles and the like, generally adopts a method that an image acquisition device is used for acquiring a surface image after the automobile coating, and since the size of the acquired surface image is large and the size of a small defect needing to be detected is small, more than 95% of the size is less than 16 pixels 16.
In order to overcome the above disadvantages, in the method for detecting defects on the surface of painted automobile based on deep learning according to the embodiment of the present invention, after the detection image of the region to be detected on the surface of painted automobile is obtained in step S1, in step S2, the detection image is first segmented by using an image processing method, and before segmentation, an appropriate segmentation size is set in combination with the limitations of detection accuracy, input requirements of a detection model, and the like, and a large image is uniformly segmented into a plurality of small images with set sizes. The above-mentioned set size is set to the size M for the sake of convenienceX，XTo a set size, i.e. M320To set the size of the small image after segmentation to 320, and M480The size of the small image after division is set to 480, and so on.
, before inputting the small image test set to the corresponding deep learning network model, the deep learning network model needs to be trained to improve the robustness of the model, specifically, each of a plurality of small images with set sizes is iteratively trained on the same deep learning network models, and the deep learning network models trained by a plurality of small images with different set sizes are different.
For example: by a plurality of M320Small image is acquired after training the deep learning network model and acquired as M320Correspondingly deeply learning the network model; then using a plurality of M480Small image is acquired after training the deep learning network model and acquired as M480 deep correspondinglyAnd (5) learning the network model. Therefore, when there are N types of set sizes, N different deep learning network models may be acquired first in the pre-training.
In addition, the method for detecting the surface defects of the automobile coating based on the deep learning provided by the embodiment of the invention limits N to be more than or equal to 2. The above approach is taken into account: due to the deep learning network models trained by the training images with different sizes, the capability of detecting defects is different, and the good complementarity is achieved. For example: when the inspection image is divided into small images, the defective portion is located in the edge area of the small image having a size of 320, but is located in the middle area of the small image having a size of 480, and is M-divided in this case320The missed defects of the deep learning network model after the training of the small images can be detected by M480And detecting the deep learning network model after the small image training. Therefore, in the embodiment of the present invention, two or more small image training sets with different sizes are used to train the deep learning network model respectively, so as to obtain different deep learning network models. During image detection, N small image test sets with different sizes are established by uniformly dividing a detected image in different modes, namely according to different set sizes.
Further , in step S3, the small images with the same set size in each small image test set are sequentially input into the corresponding deep learning network model to obtain the detection result corresponding to each small image, and finally a detection result list is formed, so that N detection result lists can be obtained for the same detection images.
Further , in step S4, the N detection result lists obtained in step S3 are subjected to de-integration processing, that is, after data of duplicates, redundancies, and significant defects are removed by integration processing, defect detection results are obtained.
According to the automobile coating surface defect detection method based on deep learning provided by the embodiment of the invention, the same detection image is divided by different set sizes, small image test sets with different sizes are obtained and are respectively input into a pre-trained deep learning network model, so that a plurality of detection result lists aiming at the same detection image are obtained, and then different detection result lists are subjected to integrated processing to obtain defect detection result information contained in the detection image.
Based on the contents of the above embodiments, as alternative embodiments, the deep learning network model is built based on YOLOv3 algorithm.
Because the YOLOv3 algorithm has a fast image processing speed and relatively high detection performance for small targets, in the method for detecting the defects of the surface of the automobile painting based on the deep learning provided by the embodiment of the invention, the deep learning network model is established based on the YOLOv3 algorithm. Since the image input size of the YOLOv3 algorithm is between 418 and 640 pixels, the output result has high accuracy, so in the method for detecting the automobile painting surface defect based on deep learning provided by the embodiment of the invention, N can be set to be 2, MXAre respectively set as M320And M480。
As shown in FIG. 2, before inputting a small image test set to a corresponding deep learning network model, firstly, image acquisition is carried out on an automobile painting surface area to obtain a plurality of training images, after image segmentation and image annotation are carried out on each training image, a small image test set with the size of 320 pixels and a small image test set with the size of 480 pixels are obtained, the deep learning network model (Darknet-M for short) is trained by using the small image test set with the size of 320 pixels to obtain an th deep learning network model (Darknet-M for short)320) And training the DarknetNet-M by using a small image test set of 480 pixels to obtain a second deep learning network model (simply referred to as: DarknetNet-M480)。
, acquiring image of the region to be detected, and dividingRespectively carrying out image segmentation on the image to be detected to obtain M320Small image set and M480A small set of images. Will M320Small image set input to DarknetNet-M320Obtaining the test result list, and adding M480Small image set input to DarknetNet-M480And processing the th detection result list and the second detection result list to obtain a defect detection result corresponding to the image to be detected.
It should be noted that, for convenience of description, only an embodiment in which N is 2 and the sizes are set to be 320 pixels and 480 pixels, respectively, is listed in fig. 2, but this is not a limitation to the scope of the embodiments of the present invention. Theoretically, when the value of N is larger, the obtained defect detection result is more accurate.
Based on the content of the foregoing embodiments, as alternative embodiments, the processing is performed on the N detection result lists, including but not limited to, performing de-duplication integration on the N detection result lists based on a non-maximization suppression algorithm (NMS).
The NMS algorithm is used for suppressing elements which are not maximum values and searching local maximum values, namely, when the characteristics of N detection result lists are extracted through a sliding window, scores are obtained every windows through classification and identification of a classifier, but the condition that a plurality of windows and other windows contain or are mostly crossed (namely, more data are crossed in the detection result lists) is caused when the characteristics of the sliding window are extracted.
Based on the content of the foregoing embodiments, as optional embodiments, before acquiring the detection image of the area to be detected on the painted surface of the automobile, the method further includes adjusting the shooting distance and adjusting the illumination intensity of the area to be detected.
When the detection image of the area to be detected on the coating surface of the automobile is obtained, if the interference of external light is large, the obtained detection image cannot well reflect the characteristics of the defect part, so that the detection result is distorted, and even the detection cannot be finished. In the embodiment of the invention, the external equipment is used for supplementing light to the area to be detected so as to counteract the interference of stray light in the uncertain environment.
, because the intensity of light supplement is different due to different shooting angles and shooting positions, the method for detecting defects on the surface of the automobile coating provided by the embodiment of the invention can achieve the purpose of providing different light supplement intensities by adjusting the illumination intensity of the area to be detected, and in addition, , the method can achieve the purpose of obtaining images to be detected with different accuracies by adjusting the shooting distance, namely, the distance between the shooting device and the area to be detected.
It should be noted that, in this embodiment, how to adjust the shooting distance and adjust the illumination intensity of the area to be detected are not specifically limited, and may be achieved manually, electrically, or pneumatically.
Based on the content of the above embodiments, as optional embodiments, before inputting the small image test set to the corresponding deep learning network model, the method further includes a process of pre-training the deep learning network model, specifically, the method includes segmenting the pre-training image of the automobile painted surface to obtain a plurality of pre-training small images of th preset size, sequentially using each pre-training small image as the input of the deep learning network model to realize iterative training of the deep learning network model, and obtaining the deep learning network model corresponding to the pre-training small image of th preset size.
When N is larger than or equal to 2, the number of the obtained deep learning network models is also N, and each deep learning network models correspond to each set size.
Based on the above description of the embodiments, as alternative embodiments, the segmentation of the pre-training image of the painted surface of the automobile includes:
calculating the segmentation grid of the pre-training image, and the column number G of the segmentation gridc＝floor(Lw/M)，Number of lines G of division gridr＝floor(Lh(ii)/M); segmenting the pre-training image by utilizing a segmentation grid to obtain a plurality of pre-training small images; calculating the actual size of the pre-trained small image, the width P of the pre-trained small imagew＝Lw/GcHeight P of the pre-trained small imageh＝Lh/Gr(ii) a Wherein floor (x) is an integer function, LWFor the width of the pre-training image, LhM is the th preset size for the height of the pre-training image.
Specifically, darknet-M is obtained still with N ═ 2320And DarknetNet-M480The description is given for the sake of example:
when pre-training images were acquired, points gave the pre-training image a size of 3856 x 2764 (i.e., L)W＝3856，Lh2764). The pre-training image pair DarknetNet-M can be utilized first320Pre-training, where M is 320, includes:
column number G of computational gridc＝floor(Lw/M) can calculate Gc12, number of rows of grid Gr＝floor(LhThe number of rows of the grid is 8, namely the pre-training image can be divided into 96 pre-training small images with 12 columns and 8 rows, the specific size of each pre-training small image is calculated respectively to complete the establishment of the detection result label of each pre-training small image, and step , the 96 pre-training small images are used as DarknetNet-M320The detection result label corresponding to each small image is taken as output to complete the DarknetNet-M320 times of training.
Similarly, the method can be adopted, and the same pre-training image is used for DarknetNet-M480A corresponding exercises were performed.
Based on the content of the foregoing embodiments, as optional embodiments, before processing the N detection result lists, the method further includes converting coordinates of an upper left point and a lower right point of the small image in each detection result list into coordinates of an upper left point and a lower right point of the detection image.
Specifically, after the small images subjected to the segmentation processing are input to the deep learning network model, the size of the acquired detection result and the size of the image to be processed are different, so that the small images are inconvenient to read and understand. On the basis, the coordinates of the upper left point and the coordinates of the lower right point of the small images in each detection result list are converted into the coordinates of the upper left point and the coordinates of the lower right point of the detection images, so that the small images in the detection result lists are converted into the same size of the detection images.
As shown in FIG. 3, the embodiment of the invention provides deep learning-based automobile painting surface defect detection systems, which include, but are not limited to, a visual inspection device 31 and an image processing unit 32, wherein the visual inspection device 31 is used for acquiring an inspection image of an area to be inspected on the automobile painting surface, and the image processing unit 32 includes a processing sub-unit 321, a second processing sub-unit 322 and a third processing sub-unit 323.
The -th processing subunit 321 is configured to uniformly divide the detection image into a plurality of small images with set sizes to construct a small image test set, the second processing subunit 322 stores deep learning network models, and is configured to input the small image test set into the deep learning network models to obtain detection result lists corresponding to the small images with the set sizes, where the set sizes are N, and N is greater than or equal to 2, the deep learning network models are obtained by training small image training samples with each set size and corresponding detection result labels, and the third processing subunit is configured to perform de-duplication integration processing on the N detection result lists to obtain defect detection results.
The embodiment of the invention also provides automobile coating surface defect detection devices based on deep learning, which comprise but are not limited to a visual detection device 31 and an image processing unit 32, wherein the visual detection device 31 comprises a wheel type chassis metal bracket 311 and an image acquisition unit 312;
the image acquisition unit 312 is fixedly mounted on a wheel-type chassis metal bracket 311, and the wheel-type chassis metal bracket 311 is used for adjusting the height and the pitch angle of the image acquisition unit 312;
the image acquisition unit 312 includes at least: the device comprises a laser range finder 3121, an area array CCD camera 3122, an LED light source 3123 and a light source brightness controller 3124.
The laser range finder 3121 is used for obtaining the distance from the area array CCD camera 3122 to the area to be detected on the automobile coating surface; the light source brightness controller 3124 is used for adjusting the brightness of the LED light source 3123 according to the distance acquired by the laser range finder 3121; the vision processing device 31 is used for acquiring a detection image of the area to be detected on the coating surface of the automobile.
The image processing unit 312 is configured to uniformly divide the detection image into a plurality of small images with a set size to construct a small image test set, input the small image test set into a deep learning network model, obtain a detection result list corresponding to the small images with the set size, where the set size is N, and N is greater than or equal to 2, perform de-duplication integration processing on the N detection result lists, and obtain a defect detection result, where the deep learning network model is obtained by training on the basis of each small image training sample with the set size and a corresponding detection result label.
TABLE 1 hardware parameter specification of automobile coating surface defect detection system based on deep learning
Figure BDA0002227841440000121
As shown in fig. 4, the visual inspection device 31 includes a wheel-type chassis metal bracket 311 and an image capturing unit 312; the image acquisition unit 312 is fixedly installed on a wheel type chassis metal bracket 311, and the wheel type chassis metal bracket 311 can be used for adjusting the height and the pitch angle of the image acquisition unit; the image acquisition unit 312 includes: the device comprises a laser range finder 3121, an area array CCD camera 3122, an LED light source 3123 and a light source brightness controller 3124.
The laser range finder 3121 is used for obtaining the distance from the area array CCD camera to the area to be detected on the automobile coating surface; the light source brightness controller 3124 is configured to adjust the brightness of the LED light source 3123 according to the distance detected by the laser range finder 3121.
Specifically, the LED light source 3123 may be a white LED light source with adjustable illumination intensity of the central hole, and the illumination intensity is controlled by the light source brightness controller 3124. And the light source controller 3124 receives the signal fed back from the laser range finder 3121, and automatically adjusts the illumination brightness of the LED light source 3123, for example, by adjusting the voltage or current input to the LED light source 3123, or by controlling a partial switch of a lamp bead in the LED light source.
As shown in table 1, the hardware parameter specifications of the components of the deep learning-based automotive painting surface defect detection system provided in the embodiment of the present invention are the parameter specifications listed in the table, but are not to be considered as limiting the scope of the embodiment of the present invention.
According to the system and the device for detecting the defects of the automobile painting surface based on the deep learning, provided by the embodiment of the invention, the same detection image is divided by using different set sizes, small image test sets with different sizes are obtained and are respectively input into corresponding deep learning network models trained in advance, so that a plurality of detection result lists aiming at the same detection image are obtained, and the defect detection result information contained in the detection image is obtained after integration processing is carried out on the different detection result lists.
Fig. 5 illustrates an entity structure diagram of electronic devices, as shown in fig. 5, the electronic device may include a processor (processor)510, a communication Interface (Communications Interface)520, a memory (memory)630 and a communication bus 540, wherein the processor 510, the communication Interface 6520 and the memory 530 complete communication with each other through the communication bus 540, the processor 510 may call logic instructions in the memory 530 to execute a method of obtaining a detection image of a region to be detected on a painted surface of an automobile, uniformly dividing the detection image into a plurality of small images with set sizes to construct a small image test set, inputting the small image test set into a deep learning network model to obtain a detection result list corresponding to the small images with the set sizes, wherein the set sizes are N, N is greater than or equal to 2, performing de-integration processing on the N detection result lists to obtain defect detection results, and wherein the deep learning network model is trained based on a small image sample with the set size of each and a corresponding detection result label.
It is to be understood that the technical solution of the present invention may be embodied in the form of a software product, or a part of the technical solution, which is stored in storage media and includes several instructions for making computer devices (which may be personal computers, servers, or network devices) execute all or part of the steps of the methods according to the embodiments of the present invention.
In another aspect, an embodiment of the present invention further provides non-transitory computer-readable storage media, on which a computer program is stored, where the computer program is implemented to perform the transmission method provided in the foregoing embodiments, where the computer program is executed by a processor to obtain a detection image of an area to be detected on a painted surface of an automobile, uniformly divide the detection image into a plurality of small images with a set size to construct a small image test set, input the small image test set into a deep learning network model to obtain a detection result list corresponding to the small images with the set size, where the set size is N, and N is greater than or equal to 2, perform de-duplication integration processing on the N detection result lists to obtain a defect detection result, and the deep learning network model is obtained by training a small image training sample with a set size of each and a corresponding detection result label.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, that is, may be located in places, or may be distributed on a plurality of network units.
Based on the understanding that the above technical solutions essentially or contributing to the prior art can be embodied in the form of a software product that can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing computer devices (which may be personal computers, servers, or network devices, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1, automobile coating surface defect detection method based on deep learning, which is characterized by comprising the following steps:
acquiring a detection image of a to-be-detected area on the coating surface of the automobile;
uniformly dividing the detection image into a plurality of small images with set sizes to build a small image test set;
inputting the small image test set into a deep learning network model, and acquiring a detection result list corresponding to the small image with the set size; wherein the set sizes are N, and N is more than or equal to 2;
performing de-duplication integration processing on the N detection result lists to obtain defect detection results;
the deep learning network model is obtained by training based on small graph training samples with the set size of every and corresponding detection result labels.
2. The deep learning-based automobile painting surface defect detection method as claimed in claim 1, wherein the deep learning network model is established based on a YOLOv3 algorithm.
3. The method for detecting the defects of the painted surface of the automobile based on the deep learning as claimed in claim 1, wherein the processing of the N detection result lists comprises:
and performing de-duplication integration on the N detection result lists based on a non-maximization inhibition algorithm.
4. The method for detecting the defects of the automobile painting surface based on the deep learning as claimed in claim 1, further comprising, before the obtaining of the detection image of the to-be-detected region of the automobile painting surface: and adjusting the shooting distance and the illumination intensity of the area to be detected.
5. The method for detecting defects of a painted surface of an automobile based on deep learning of claim 1, further comprising a process of pre-training the deep learning network model before inputting the small image test set to the corresponding deep learning network model, specifically:
segmenting the pre-training image of the automobile coating surface to obtain a plurality of pre-training small images of th preset size;
and sequentially taking each pre-training small image as the input of the deep learning network model, realizing iterative training of the deep learning network model, and acquiring the deep learning network model corresponding to the pre-training small image with the th preset size.
6. The method for detecting defects of a painted surface of an automobile based on deep learning of claim 5, wherein segmenting the pre-training image of the painted surface of the automobile comprises:
calculating a segmentation grid of the pre-training image, the column number G of the segmentation gridc＝floor(Lw/M), the number of rows G of the segmentation gridr＝floor(Lh/M)；
Segmenting the pre-training image by using the segmentation grid to obtain a plurality of pre-training small images;
calculating the actual size of the pre-training small image, wherein the width P of the pre-training small imagew＝Lw/GcHeight P of said pre-trained small imageh＝Lh/Gr；
Wherein floor (x) is an integer function, LWFor the width of the pre-training image, LhM is the th preset size for the height of the pre-training image.
7. The method for detecting the defects of the painted surface of the automobile based on the deep learning as claimed in claim 3, further comprising, before the processing of the N lists of the detection results:
and converting the coordinates of the upper left point and the coordinates of the lower right point of the small images in each detection result list into the coordinates of the upper left point and the coordinates of the lower right point of the detection images.
8, A system for detecting defects of automobile coating surface based on deep learning, which is characterized by comprising a visual detection device and an image processing unit;
the visual detection device is used for acquiring a detection image of a to-be-detected area on the coating surface of the automobile;
the image processing unit comprises an th processing sub-unit, a second processing sub-unit and a third processing sub-unit;
the th processing subunit is used for uniformly dividing the detection image into a plurality of small images with set sizes so as to construct a small image test set;
the second processing subunit is stored with a deep learning network model and is used for inputting the small image test set into the deep learning network model and acquiring a detection result list corresponding to the small image with the set size; wherein the set sizes are N, and N is more than or equal to 2;
the deep learning network model is obtained by training small graph training samples with the set size of every and corresponding detection result labels, and the third processing subunit is used for performing de-duplication integration processing on the N detection result lists to obtain defect detection results.
9, automobile coating surface defect detection devices based on deep learning, which is characterized by comprising a visual detection device and an image processing unit, wherein the visual detection device comprises a wheel type chassis metal bracket and an image acquisition unit;
the image acquisition unit is fixedly arranged on the wheel type chassis metal support, and the wheel type chassis metal support is used for adjusting the height and the pitch angle of the image acquisition unit;
the image acquisition unit includes: the system comprises a laser range finder, an area array CCD camera, an LED light source and a light source brightness controller;
the laser range finder is used for acquiring the distance from the area array CCD camera to the area to be detected on the automobile coating surface; the light source brightness controller is used for adjusting the brightness of the LED light source according to the distance;
the vision processing device is used for acquiring a detection image of a to-be-detected area on the coating surface of the automobile;
the image processing unit is used for uniformly dividing the detection image into a plurality of small images with set sizes so as to construct a small image test set; inputting the small image test set into a deep learning network model, and acquiring a detection result list corresponding to the small image with the set size; wherein the set sizes are N, and N is more than or equal to 2; performing de-duplication integration processing on the N detection result lists to obtain defect detection results;
the deep learning network model is obtained by training based on small graph training samples with the set size of every and corresponding detection result labels.
10, electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the deep learning based automotive painting surface defect detection method of any of claims 1 to 7 through .
CN201910957565.1A 2019-10-10 2019-10-10 automobile coating surface defect detection method and system based on deep learning Pending CN110738644A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910957565.1A CN110738644A (en) 2019-10-10 2019-10-10 automobile coating surface defect detection method and system based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910957565.1A CN110738644A (en) 2019-10-10 2019-10-10 automobile coating surface defect detection method and system based on deep learning

Publications (1)

Publication Number Publication Date
CN110738644A true CN110738644A (en) 2020-01-31

Family

ID=69268470

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910957565.1A Pending CN110738644A (en) 2019-10-10 2019-10-10 automobile coating surface defect detection method and system based on deep learning

Country Status (1)

Country Link
CN (1) CN110738644A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111487250A (en) * 2020-04-02 2020-08-04 苏州奥创智能科技有限公司 Intelligent visual detection method and system applied to injection molding defective product detection
CN113362194A (en) * 2020-03-04 2021-09-07 丰田自动车株式会社 Coating quality prediction device and method for generating learned model
CN113824951A (en) * 2021-09-20 2021-12-21 苏州凯仕弘科技有限公司 Camera module visual detection system
CN115684172A (en) * 2022-10-09 2023-02-03 迁安市福运机动车检测有限公司 Automobile appearance detection system and using method thereof
CN117269168A (en) * 2023-09-15 2023-12-22 昆山精诚得精密五金模具有限公司 New energy automobile precision part surface defect detection device and detection method

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100124600A1 (en) * 2008-11-17 2010-05-20 Basol Bulent M Method and apparatus for detecting and passivating defects in thin film solar cells
CN107977961A (en) * 2017-11-24 2018-05-01 常州大学 Textile flaw detection method based on peak value coverage values and composite character
CN108765430A (en) * 2018-05-24 2018-11-06 西安思源学院 A kind of heart left chamber region segmentation method based on cardiac CT image and machine learning
CN109239073A (en) * 2018-07-28 2019-01-18 西安交通大学 A kind of detection method of surface flaw for body of a motor car
CN109815800A (en) * 2018-12-17 2019-05-28 广东电网有限责任公司 Object detection method and system based on regression algorithm

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100124600A1 (en) * 2008-11-17 2010-05-20 Basol Bulent M Method and apparatus for detecting and passivating defects in thin film solar cells
CN107977961A (en) * 2017-11-24 2018-05-01 常州大学 Textile flaw detection method based on peak value coverage values and composite character
CN108765430A (en) * 2018-05-24 2018-11-06 西安思源学院 A kind of heart left chamber region segmentation method based on cardiac CT image and machine learning
CN109239073A (en) * 2018-07-28 2019-01-18 西安交通大学 A kind of detection method of surface flaw for body of a motor car
CN109815800A (en) * 2018-12-17 2019-05-28 广东电网有限责任公司 Object detection method and system based on regression algorithm

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
FEI CHANG等: "A Mobile Vision Inspection System for Tiny Defect Detection of Smooth Car-body Surface Based on Deep Ensemble Learning", 《MEASUREMENT SCIENCE AND TECHNOLOGY》 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113362194A (en) * 2020-03-04 2021-09-07 丰田自动车株式会社 Coating quality prediction device and method for generating learned model
CN113362194B (en) * 2020-03-04 2024-05-03 丰田自动车株式会社 Coating quality prediction device and method for generating learned model
CN111487250A (en) * 2020-04-02 2020-08-04 苏州奥创智能科技有限公司 Intelligent visual detection method and system applied to injection molding defective product detection
CN113824951A (en) * 2021-09-20 2021-12-21 苏州凯仕弘科技有限公司 Camera module visual detection system
CN115684172A (en) * 2022-10-09 2023-02-03 迁安市福运机动车检测有限公司 Automobile appearance detection system and using method thereof
CN117269168A (en) * 2023-09-15 2023-12-22 昆山精诚得精密五金模具有限公司 New energy automobile precision part surface defect detection device and detection method
CN117269168B (en) * 2023-09-15 2024-04-09 昆山精诚得精密五金模具有限公司 New energy automobile precision part surface defect detection device and detection method

Similar Documents

Publication Publication Date Title
CN110738644A (en) automobile coating surface defect detection method and system based on deep learning
CN111340752B (en) Screen detection method and device, electronic equipment and computer readable storage medium
CN109871895B (en) Method and device for detecting defects of circuit board
CN107941808B (en) 3D printing forming quality detection system and method based on machine vision
CN111612737B (en) Artificial board surface flaw detection device and detection method
CN103617625B (en) Image matching method and image matching device
CN110634140A (en) Large-diameter tubular object positioning and inner wall defect detection method based on machine vision
US20210150700A1 (en) Defect detection device and method
CN111222395A (en) Target detection method and device and electronic equipment
CN112505056A (en) Defect detection method and device
CN112819772A (en) High-precision rapid pattern detection and identification method
CN111598913B (en) Image segmentation method and system based on robot vision
CN111721259A (en) Underwater robot recovery positioning method based on binocular vision
CN102901444A (en) Method for detecting component size based on matching pursuit (MP) wavelet filtering and detecting system thereof
CN107369176B (en) System and method for detecting oxidation area of flexible IC substrate
CN114926407A (en) Steel surface defect detection system based on deep learning
CN112750113B (en) Glass bottle defect detection method and device based on deep learning and linear detection
US20230053085A1 (en) Part inspection system having generative training model
CN115830018B (en) Carbon block detection method and system based on deep learning and binocular vision
CN113705351A (en) Vehicle damage assessment method, device and equipment
CN108154496B (en) Electric equipment appearance change identification method suitable for electric power robot
CN113066088A (en) Detection method, detection device and storage medium in industrial detection
CN116071315A (en) Product visual defect detection method and system based on machine vision
CN109622404B (en) Automatic sorting system and method for micro-workpieces based on machine vision
CN109785290B (en) Steel plate defect detection method based on local illumination normalization

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200131