CN113808080B - Method for detecting number of interference fringes of glass panel of camera hole of mobile phone - Google Patents

Method for detecting number of interference fringes of glass panel of camera hole of mobile phone Download PDF

Info

Publication number
CN113808080B
CN113808080B CN202110922393.1A CN202110922393A CN113808080B CN 113808080 B CN113808080 B CN 113808080B CN 202110922393 A CN202110922393 A CN 202110922393A CN 113808080 B CN113808080 B CN 113808080B
Authority
CN
China
Prior art keywords
area
glass panel
interference fringe
mobile phone
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110922393.1A
Other languages
Chinese (zh)
Other versions
CN113808080A (en
Inventor
李一芒
刘康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changzhou University
Original Assignee
Changzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changzhou University filed Critical Changzhou University
Priority to CN202110922393.1A priority Critical patent/CN113808080B/en
Publication of CN113808080A publication Critical patent/CN113808080A/en
Application granted granted Critical
Publication of CN113808080B publication Critical patent/CN113808080B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20024—Filtering details
    • G06T2207/20032—Median filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30242—Counting objects in image

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Quality & Reliability (AREA)
  • Geometry (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention relates to the technical field of interference image processing, in particular to a method for detecting the number of interference fringes of a glass panel of a camera hole of a mobile phone, which comprises the following steps: s1, acquiring interference fringe images of a glass panel of a camera hole of a mobile phone, and carrying out data enhancement and labeling; s2, training, tuning and accuracy verification of the YOLOv2 network model; s3, inputting an interference fringe image to be detected into a YOLOv2 network model, and detecting an interference fringe image; s4, the length and the width of the detected target area are enlarged according to a certain proportion and then intercepted and analyzed; s5, preprocessing the interference fringe pattern of the cut target area, and obtaining the number of interference fringes through the area of the communication area. According to the invention, through collecting interference fringe images of the glass panel of the camera hole of the mobile phone, the operation steps are simplified by utilizing the YOLOv2 target detection algorithm, and the instantaneity is improved; and the detected stripe area is calculated by adopting a scanning line seed filling graphics method, so that the real-time performance is high.

Description

Method for detecting number of interference fringes of glass panel of camera hole of mobile phone
Technical Field
The invention relates to the technical field of interference image processing, in particular to a method for detecting the number of interference fringes of a glass panel of a camera hole of a mobile phone.
Background
In the daily life of modern people, numerous electronic display devices have become necessary, such as smart phones, tablet computers, flat televisions, PCs and other devices, and display screens play a vital role in these devices, so that the good display effect can promote the use feeling of users. At present, the glass panel is mostly covered outside the display screen of various electronic equipment, so that the touch texture can be improved, and the screen can be protected. The glass panel has high surface hardness, high strength, good transmittance and good touch feeling. Glass panels, due to their good characteristics, are chosen as the screen material by most electronic display manufacturers. Because the glass panel has high hardness and high strength, even bending-resistant glass materials are present, the glass panel is very suitable for producing glass toughened films so as to protect a screen, the surface quality of the glass is critical, and defects on the surface of the glass can influence the look and feel of people. Various electronic display screen production technologies are rapidly developed, the manufacturing precision is increasingly severe, and the detection standard is correspondingly and greatly increased. Because of the great demands on glass panels, quality inspection is critical, and there are many defects of glass panels, such as point defects, edge chipping, pits, scratches, foreign materials, mottle, and surface defects.
The touch screen surface of smart phones currently on the market is covered with a glass panel under which various elements of the phone such as touch screen, camera and various optical devices are placed. With the increasing demand of users for large-sized screen mobile phones, especially the appearance of 2.5D screens popular in recent years, the layout of elements such as mobile phone cameras is led to be closer to the edge of the screen, and when 2.5D glass panels are processed, the processing yield of the cambered surface of the edge of the glass panels is not high, and finally the accuracy of the surface type of the glass panels corresponding to the positions of optical sensors of the mobile phone cameras is not high, so that the photographing effect of the mobile phones is seriously affected after the mobile phones are assembled. The most effective means for detecting the quality of the glass panel of the camera hole of the mobile phone is to adopt a laser interferometer, and judge the degree of influence on the imaging quality of the mobile phone after the camera hole of the mobile phone is processed by detecting the number and the quality of interference fringes of the glass panel, so that the inspection of the processing quality of the glass panel part of the camera hole of the mobile phone is completed.
However, when the domestic laser interferometer detects the number of stripes of the glass panel of the camera hole of the mobile phone, the glass panel needs to be fixed at a detection position, so that interference fringe images can be conveniently collected at fixed points for further analysis and detection.
In addition, a method for reading the number of fringes is common in the prior art, for example (Mei Qisheng, wang Min, zhou Qun. A detection method of a multi-noise interference fringe image), an image processing technology is utilized to perform related preprocessing on the image, then edge features are extracted, and then a central line and feature points are extracted and curve fitting is performed, so as to obtain the number of fringes. For example, (CN 201310003928.0 newton ring interference fringe counting device and counting method) uses hardware processing technology to detect the variation of the light intensity and shade by hardware circuit, and then converts it into electric signal, and then processes the electric signal to obtain fringe number.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: the interference fringe image of the glass panel of the camera hole of the mobile phone is collected, and the YOLOv2 target detection algorithm is utilized, so that the operation steps are simplified, and the instantaneity is improved; and aiming at the detected stripe region, the scanning line seed filling graphics method is adopted to calculate the stripe number, so that the detection efficiency is high.
The invention adopts the technical scheme that: a method for detecting the number of interference fringes of a glass panel with a camera hole of a mobile phone comprises the following steps:
s1, collecting interference fringe images of a certain number of glass panels of camera holes of a mobile phone, carrying out data enhancement on the interference fringe images, marking by using marking software, constructing a sample data set, and dividing the sample data set into a training set, a verification set and a test set according to a proportion;
s1 comprises the following steps:
s11, acquiring interference fringe patterns of different types (the sizes and the shapes of mobile phone camera holes of different manufacturers are different, so that the mobile phone camera hole data of different types and the mobile phone camera hole glass panels with different precision are acquired through a Fizeau laser interferometer (the precision refers to the precision of the camera hole glass panels, the fringe number can reflect the precision of the glass panels, and therefore the fringe number of each camera hole glass panel is different), deleting the interference fringe patterns with poor quality (the images are blurred or target areas are not completely placed in a detection range, and image defects appear), and finishing data cleaning;
the Fizeau laser interferometer can be called an optical flat plate and is used for detecting a glass panel of a camera hole of a mobile phone; the surface of the workpiece is subjected to grinding or polishing processing by utilizing the Fizeau laser interferometer to test, so that the reflected light on the surface of the workpiece has enough intensity to interfere with the light reflected by the action surface of the Fizeau laser interferometer to form a color band;
s12, the data enhancement method is to expand the cleaned interference fringe image data set by one time in a translation and turnover mode;
s13, marking an interference fringe pattern of a target area (such as a circular area in FIG. 2) by using a LabelImg marking tool, wherein a label is hole (name of the target area), and each picture generates an xml file;
s2, using a training set for training the YOLOv2 network model, using a verification set for parameter tuning of the YOLOv2 network model, and using a test set for accuracy verification of the network model;
s21, carrying out K-means dimension clustering on a boundary frame of the interference fringe pattern of the glass panel of the camera hole; the IOU value of the boundary box and the clustering center is used as a distance index, and the IOU value is the ratio of the intersection area of the boundary box and the clustering center to the union area of the clustering center and the boundary box; taking the iterative clustering center as a priori frame; the distance index calculating method comprises the following steps: d (box, centroid) =1-IOU (box, centroid);
s22, 32 layers of a YOLOv2 network model structure based on a Darkent-19 network structure comprise 23 convolution layers, 5 maximum pooling layers, 2 route layers, 1 reorg layer and an output layer;
s23, modifying an initial YOLOv2 network model structure aiming at detection of interference fringe areas of a glass panel with camera holes, and modifying the structure into a single-target detection network and single-class output; modifying the dimension size of the output of the YOLOv2 network model: sx S x (K x (5+C)), where S is the number of lattice areas divided by the picture, K is the number of prior frames of the cluster, and C is the number of categories;
the YOLOv2 network model output contains 5 parameters predicted for each target box: t is t x 、t y 、t w 、t h 、t 0 The coordinates of the prediction frame are predicted as follows:
b x ＝σ(t x )+c x
b y ＝σ(t y )+c y
wherein (c) x ,c y ) For the upper right corner coordinates of the corresponding picture grid, (b) x ,b y ) To predict the frame center coordinates, p w And p h Is the width and height of the prior frame, t x 、t y For predicting relative parameters of frame center, t w 、t h To predict the relative parameters of the frame width and the frame height, t 0 Is a relevant parameter of confidence.
The actual positions of the prediction frames in the picture are:
x＝(b x /S)×W
y＝(b y /S)×H
w＝(b w /S)×W
h＝(b h /S)×H
wherein x, y, w, h is the actual center coordinates of the prediction frame and the width and height, S is the number of grid areas divided by the picture, and W, H is the actual width and height of the picture;
s24, performing non-maximum suppression on the output target detection frame, and screening out an optimal target detection frame;
the non-maximum value suppression method of S24 comprises the following steps:
s241, sequencing all candidate frames output by the YOLOv2 network model according to the confidence level, and setting the candidate frame with the highest confidence level as a target frame;
s242, traversing all candidate frames except the target frame, and deleting the target frame if the IOU values of the candidate frames and the target frame are greater than a threshold value;
s243, re-selecting the candidate frame with the highest confidence from the rest candidate frames as a new target frame, repeating S242 until all the candidate frames are screened, and finally outputting all the target frames.
S25, calculating and back-propagating an output result and an actual value of the YOLOV2 network model by using a loss function, and updating weight parameters of a network structure;
s3, inputting the interference fringe image in the test set into a YOLOv2 network model, and detecting an interference fringe image of the glass panel in the camera hole area of the mobile phone;
s4, expanding the length and width of the detected target area according to a certain proportion, and then intercepting and analyzing;
in order to ensure that the output bounding box can contain the fringe pattern information of the target area as much as possible, the bounding box of the detected target area is enlarged, the length and the width of the output bounding box are enlarged by X times, and the enlargement factors are determined according to the actual detection effect of the YOLOv2 network model;
s5, preprocessing the interference fringe pattern of the cut target area, and obtaining the number of interference fringes of the glass panel with the camera hole by analyzing the area of the image communication area;
the method comprises the following specific steps:
s51, median filtering is carried out on the fringe patterns, the filtering method is that each image is scanned line by line, when each image is processed, whether the pixel is an extremum under a filtering window is judged, if yes, the pixel is processed by median filtering, and if not, the pixel is not processed;
s52, in order to reduce interference of the background on the stripe image, according to the size of the target frame, taking a target center point as a round dot, taking the diameter as the length of the shortest side of the target frame, reserving pixel points in the round dot, and setting the rest pixel points as 255 as the background;
s53, binarizing the processed fringe pattern, and performing multiple closing operation operations, namely expanding the image first and then corroding, connecting the fringes broken by noise, and reserving fringe information to the maximum extent;
s54, scanning and filling the connected domain by using a scanning line seed filling method to obtain the area of each stripe connected domain; and (3) sorting the areas of the connected domains in a descending order, wherein the largest connected domain area is used as a first connected domain, and if the area of the previous connected domain is more than three times that of the current screened connected domain and the area of the previous connected domain is smaller than a threshold value, neglecting the subsequent connected domains, and obtaining the number of stripes according to the number of the residual connected domains.
The scanning line seed filling method comprises the following steps:
s541, initializing an empty stack, and pushing seed points (x 1, x 2) into the stack; judging whether the stack is empty, ending the algorithm if the stack is empty, otherwise, taking out stack top elements as seed points (x 1, x 2) of the current scanning line, wherein x2 is the current scanning line;
s542, starting from the seed points (x 1, x 2), filling along the left and right directions of the current scanning line until the boundary, and respectively marking the left and right end point coordinates of the section as left and right;
s543, respectively checking pixels of two scanning lines x2-1 and x2+1 adjacent to the current scanning line in a section [ left, right ], searching from left to right, if non-boundary unfilled pixel points exist, finding out the rightmost one of the adjacent pixel points, taking the rightmost one as a seed point to be pushed into a stack, and returning to the step S542.
The invention has the beneficial effects that: according to the method for detecting the number of interference fringes of the glass panel of the camera hole of the mobile phone, a detected sample is not required to be fixed, and a positioning target area can be detected for analysis; the method for detecting the target area is based on a YOLOv2 target detection algorithm, and can meet the real-time requirement while automatically detecting; the number of stripes is calculated by a pattern method such as a scanning line seed filling method for the detected stripe region.
Drawings
FIG. 1 is a basic flow chart of a detection method of the number of interference fringes of a glass panel of a camera hole of a mobile phone;
FIG. 2 is an interference fringe pattern of a target area detected by the YOLOv2 model of the present invention;
FIG. 3 is a process of performing stripe counting on the extracted stripe pattern of the present invention.
Detailed Description
The invention will be further described with reference to the accompanying drawings and examples, which are simplified schematic illustrations showing only the basic structure of the invention and thus showing only those constructions that are relevant to the invention.
The embodiment of the invention provides a method for detecting the number of interference fringes of a glass panel with a mobile phone camera, as shown in fig. 1, S1, collecting a certain number of interference fringe images of the glass panel with the mobile phone camera, carrying out data enhancement on the interference fringe images, marking by using marking software, constructing a sample data set, and dividing the sample data set into a training set, a verification set and a test set according to a ratio of 6:2:2;
s11, collecting interference fringe patterns of mobile phone camera hole glass panels with different types (the types are different in shape and size of the glass panels passing through the camera hole positions) and different accuracies (the accuracies of the camera hole glass panels are indicated, namely the fringe number), deleting the interference fringe patterns with poor quality, and finishing data cleaning;
s12, the data enhancement method is to expand the cleaned interference fringe pattern data set by one time in a translation and turnover mode, wherein the size of the interference fringe pattern is 720 multiplied by 576 multiplied by 3;
s13, marking the interference fringe pattern of the target area by using a LabelImg marking tool, wherein the label is hole, and each picture generates an xml file;
for example: the image size is modified to 416 multiplied by 3, the interference fringe image of the target area is marked by using a marking tool LabelImg, the label is hole, each image generates an xml file, the image set is divided into a training set, a verification set and a test set according to proportion, wherein the training set is 7500, the verification set is 2500, the test set is 2500, and each image contains one to two target images.
S2, using a training set for training the YOLOv2 network model, using a verification set for parameter tuning of the YOLOv2 network model, and using a test set for accuracy verification of the network model;
s21, carrying out K-means dimension clustering according to the divided training set and the corresponding xml file, selecting an optimal K value, selecting a clustering center K=3, wherein the length and width of 3 clustering frames are respectively (1.84375,2.1875), (2.0625,2.53125) and (2.21875,2.09375), the maximum value of the width and the height is 13, and the IOU value of the boundary frame and the clustering center is used as a distance index during clustering analysis, wherein the IOU value is the ratio of the intersection area of the boundary frame and the clustering center to the union area of the boundary frame. Taking the iterative clustering center as a priori frame, the distance index calculation method comprises the following steps: d (box, centroid) =1-IOU (box, centroid).
S22, 32 layers of a YOLOv2 network model structure based on a Darkent-19 network structure comprise 23 convolution layers, 5 maximum pooling layers, 2 route layers, 1 reorg layer and an output layer;
wherein the feature extraction is performed using a 3 x 3 convolution kernel, the data reduction and data fusion is performed using a 1 x1 convolution kernel, and the downsampling is performed using a 2 x2 convolution kernel, a step size of 2, and a maximum pooling layer.
S23, aiming at detection of interference fringe areas of a glass panel with camera holes, an initial YOLOv2 network model structure is improved, and the structure is modified into a single-target detection network and single-class output; modifying the dimension size of the output of the YOLOv2 network model: sx s× (k× (5+C)), where s=13 is the number of lattice areas divided by the picture, k=3 is the number of prior frames of the cluster, and c=1 is the number of categories.
S24, performing non-maximum suppression on the output target detection frame, and screening out an optimal target detection frame; screening an optimal target detection frame according to the cross ratio to serve as a prediction frame;
s25, calculating and counter-propagating a predicted frame and an actual frame output by the YOLOV2 network model by using a loss function, and obtaining a trained YOLOv2 model by adopting a mini-batch gradient descent method, wherein the batch size is 64, the initial learning rate is 0.001, the weight attenuation is 0.005 and the momentum is 0.8;
fig. 2 shows the detection effect of the yolov2 model after training, 4 graphs represent the detection effect of cameras with different mobile phone glass panels, and the numerical values on the boxes in the graphs are confidence degrees.
S4, expanding the length and width of the detected target area according to a certain proportion, and then intercepting and analyzing;
in order to ensure that the output bounding box can contain the fringe pattern information of the target area as much as possible, the bounding box of the detected target area is enlarged, the length and the width of the output are enlarged by X times, and X=1.125 is set according to the actual model expression effect;
s5, preprocessing the interference fringe pattern of the cut target area, and obtaining the number of interference fringes of the glass panel with the camera hole by analyzing the area of the image communication area;
s51, median filtering is carried out on the fringe patterns, the size of the selected field is 5 multiplied by 5, the filtering method is that each image is scanned line by line, when each image is processed, whether the pixel is an extremum under a filtering window is judged, if yes, the pixel is processed by median filtering, and if not, the pixel is not processed;
s52, in order to reduce interference of the background on the stripe image, according to the size of the target frame, taking a target center point as a round dot, taking the diameter as the length of the shortest side of the target frame, reserving pixel points in the round dot, and setting the rest pixel points as 255 as the background;
s53, binarizing the processed fringe pattern, and performing multiple closing operation operations, namely expanding the image first and then corroding, connecting the fringes broken by noise, and reserving fringe information to the maximum extent; the stripe information can be retained to the greatest extent by carrying out 2 closing operations after inspection.
S54, scanning and filling the connected domain by using a scanning line seed filling method to obtain the area of each stripe connected domain; and (3) sorting the areas of the connected domains in a descending order, wherein the largest connected domain area is used as a first connected domain, if the area of the previous connected domain is more than three times that of the currently screened connected domain and the area of the previous connected domain is smaller than a threshold value, setting the threshold value as 35, neglecting the subsequent connected domains, and obtaining the number of stripes according to the number of the residual connected domains.
As shown in fig. 3, the detected target area is enlarged according to a certain proportion and then intercepted, the intercepted interference fringe pattern is subjected to related pretreatment, the areas of all connected domains in the interference fringe pattern are obtained through a scanning line seed filling method, the detected fringe pattern connected domain areas are [259, 144, 61,2,1] after being sequenced in a descending order, the largest pixel area 259 is taken as the area of the first connected domain, the area of the next connected domain is 144, three times of the area of the next connected domain is larger than the area of the previous connected domain, so that the number of the next connected domain is equal to the number of the connected domain, the connected domain area 61 can be counted, the connected domain area 2 does not meet the condition, and the remaining connected domain areas are ignored completely, so that the number of the counted connected domain areas is 3.
In fig. 3, fig. 2 is an effect diagram obtained by enlarging the target area in S4 and then cutting the target area, fig. 3 is an effect diagram obtained by preprocessing the interference fringe pattern of the target area cut in S5, and fig. 4 is an effect diagram obtained by performing the closing operation and the connected domain counting in S53.
Compared with the traditional method for counting interference fringes of the glass panel with the mobile phone camera holes, the method does not need to fix the workpiece to be detected, can detect a plurality of mobile phone camera holes at the same time, and has high instantaneity, and the counting of each fringe pattern only needs about 0.5 s.
The invention has the beneficial effects that: the method for detecting the number of interference fringes of the glass panel of the camera hole of the mobile phone can detect and locate a target area for analysis without fixing a detected sample; the method for detecting the target area is based on a YOLOv2 target detection algorithm, and can meet the real-time requirement while automatically detecting; the number of stripes is calculated by a pattern method such as a scanning line seed filling method for the detected stripe region.
With the above-described preferred embodiments according to the present invention as an illustration, the above-described descriptions can be used by persons skilled in the relevant art to make various changes and modifications without departing from the scope of the technical idea of the present invention. The technical scope of the present invention is not limited to the description, but must be determined according to the scope of claims.

Claims (4)

1. The method for detecting the number of interference fringes of the glass panel of the camera hole of the mobile phone is characterized by comprising the following steps:
s1, acquiring interference fringe images of a glass panel of a camera hole of a mobile phone, performing data enhancement and labeling on the interference fringe images, constructing a sample data set, and dividing the sample data set into a training set, a verification set and a test set according to a proportion;
the step S1 includes:
s11, collecting interference fringe patterns of different types and different precision of mobile phone camera hole glass panels through a Fizeau laser interferometer, removing interference fringe patterns with unqualified quality, and finishing data cleaning;
s12, the data enhancement method is to expand the cleaned interference fringe image data set by one time in a translation and turnover mode;
s13, marking the interference fringe pattern of the target area by using a LabelImg marking tool;
s2, using a training set for training the YOLOv2 network model, using a verification set for parameter tuning of the YOLOv2 network model, and using a test set for accuracy verification of the network model;
the step S2 includes:
s21, carrying out K-means dimension clustering on a boundary frame of the interference fringe pattern of the glass panel of the camera hole; the IOU value of the boundary box and the clustering center is used as a distance index, and the IOU value is the ratio of the intersection area of the boundary box and the clustering center to the union area of the clustering center and the boundary box; taking the iterative clustering center as a priori frame; the distance index calculating method comprises the following steps: d (box, centroid) =1-IOU (box, centroid);
s22, a YOLOv2 network model structure based on a Darkent-19 network structure comprises a convolution layer, a maximum pooling layer, a route layer, a reorg layer and an output layer;
s23, modifying an initial YOLOv2 network model structure aiming at detection of interference fringe areas of a glass panel with camera holes, and modifying the structure into a single-target detection network and single-class output; modifying the dimension size of the output of the YOLOv2 network model: sx S x (K x (5+C)), where S is the number of lattice areas divided by the picture, K is the number of prior frames of the cluster, and C is the number of categories;
s24, performing non-maximum suppression on the output target detection frame, and screening out an optimal target detection frame;
s25, calculating and back-propagating an output result and an actual value of the YOLOV2 network model by using a loss function, and updating weight parameters of a network structure;
s3, inputting an interference fringe image to be detected of the training set into a YOLOv2 network model, and detecting an interference fringe image of the glass panel in the camera hole area of the mobile phone;
s4, expanding the length and width of the detected target area according to a certain proportion, and then intercepting and analyzing;
in order to ensure that the output bounding box can contain the fringe pattern information of the target area as much as possible, the bounding box of the detected target area is enlarged, the length and the width of the output bounding box are enlarged by X times, and the enlargement factors are determined according to the actual detection effect of the YOLOv2 network model;
s5, preprocessing the interference fringe pattern of the cut target area, and obtaining the number of interference fringes of the glass panel with the camera hole by analyzing the area of the image communication area.
2. The method for detecting the number of interference fringes of a glass panel with a camera hole of a mobile phone according to claim 1, wherein the method for suppressing the non-maximum value of S24 comprises:
s241, setting all candidate frames output by the network model as target frames, wherein the candidate frame with the highest confidence level is set;
s242, traversing all candidate frames except the target frame, and deleting the target frame if the IOU value of the candidate frames and the target frame is larger than a threshold value;
s243, re-selecting the candidate frame with the highest confidence from the rest candidate frames as a new target frame, repeating S242 until all the candidate frames are screened, and finally outputting all the target frames.
3. The method for detecting the number of interference fringes of a glass panel with a camera hole of a mobile phone according to claim 1, wherein the step S5 includes:
s51, carrying out self-adaptive median filtering on the fringe patterns, wherein the filtering method is that each image is scanned line by line, judging whether a pixel is an extremum under a filtering window or not when each image is processed, if so, adopting median filtering to process the pixel, and if not, carrying out pretreatment;
s52, in order to reduce interference of the background on the stripe image, according to the size of the target frame, taking a target center point as a round dot, taking the diameter as the length of the shortest side of the target frame, reserving pixel points in the round dot, and setting the rest pixel points as 255 as the background;
s53, binarizing the processed fringe pattern, and performing multiple closing operation operations, namely expanding the image first and then corroding, connecting the fringes broken by noise, and reserving fringe information to the maximum extent;
s54, scanning and filling the connected domain by using a scanning line seed filling method to obtain the area of each stripe connected domain; and (3) sorting the areas of the connected domains in a descending order, wherein the largest connected domain area is used as a first connected domain, and if the area of the previous connected domain is more than three times that of the current screened connected domain and the area of the previous connected domain is smaller than a threshold value, neglecting the subsequent connected domains, and obtaining the number of stripes according to the number of the residual connected domains.
4. The method for detecting the number of interference fringes of a glass panel with a camera hole of a mobile phone according to claim 3, wherein the scanning line seed filling method of S54 includes:
s541, initializing an empty stack, and pushing seed points (x 1, x 2) into the stack; judging whether the stack is empty, ending the algorithm if the stack is empty, otherwise, taking out stack top elements as seed points (x 1, x 2) of the current scanning line, wherein x2 is the current scanning line;
s542, starting from the seed points (x 1, x 2), filling along the left and right directions of the current scanning line until the boundary, and respectively marking the left and right end point coordinates of the section as left and right;
s543, respectively checking pixels of two scanning lines x2-1 and x2+1 adjacent to the current scanning line in a section [ left, right ], searching from left to right, if non-boundary unfilled pixel points exist, finding out the rightmost one of the adjacent pixel points, taking the rightmost one as a seed point to be pushed into a stack, and returning to the step S542.
CN202110922393.1A 2021-08-12 2021-08-12 Method for detecting number of interference fringes of glass panel of camera hole of mobile phone Active CN113808080B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110922393.1A CN113808080B (en) 2021-08-12 2021-08-12 Method for detecting number of interference fringes of glass panel of camera hole of mobile phone

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110922393.1A CN113808080B (en) 2021-08-12 2021-08-12 Method for detecting number of interference fringes of glass panel of camera hole of mobile phone

Publications (2)

Publication Number Publication Date
CN113808080A CN113808080A (en) 2021-12-17
CN113808080B true CN113808080B (en) 2023-10-24

Family

ID=78942782

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110922393.1A Active CN113808080B (en) 2021-08-12 2021-08-12 Method for detecting number of interference fringes of glass panel of camera hole of mobile phone

Country Status (1)

Country Link
CN (1) CN113808080B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11287629A (en) * 1998-04-03 1999-10-19 Mitsutoyo Corp Optical interference type measuring device and machining device with measuring function, having the same
CN103411688A (en) * 2013-08-19 2013-11-27 广东外语外贸大学 Michelson interference ring long-distance measurement and control method and measurement and control system thereof
CN110232677A (en) * 2019-05-23 2019-09-13 天津大学 A kind of cell phone application for interference fringe analysis
AU2020101011A4 (en) * 2019-06-26 2020-07-23 Zhejiang University Method for identifying concrete cracks based on yolov3 deep learning model
CN112270252A (en) * 2020-10-26 2021-01-26 西安工程大学 Multi-vehicle target identification method for improving YOLOv2 model

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11287629A (en) * 1998-04-03 1999-10-19 Mitsutoyo Corp Optical interference type measuring device and machining device with measuring function, having the same
CN103411688A (en) * 2013-08-19 2013-11-27 广东外语外贸大学 Michelson interference ring long-distance measurement and control method and measurement and control system thereof
CN110232677A (en) * 2019-05-23 2019-09-13 天津大学 A kind of cell phone application for interference fringe analysis
AU2020101011A4 (en) * 2019-06-26 2020-07-23 Zhejiang University Method for identifying concrete cracks based on yolov3 deep learning model
CN112270252A (en) * 2020-10-26 2021-01-26 西安工程大学 Multi-vehicle target identification method for improving YOLOv2 model

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于CCD采集的Math-Zehnder干涉条纹图的处理算法;杨利红;施浣芳;陈智利;方勇;应用光学;第26卷(第2期);全文 *

Also Published As

Publication number Publication date
CN113808080A (en) 2021-12-17

Similar Documents

Publication Publication Date Title
CN113688807B (en) Self-adaptive defect detection method, device, recognition system and storage medium
CN105894036B (en) A kind of characteristics of image template matching method applied to mobile phone screen defects detection
CN109508664B (en) Dish identification pricing method based on deep learning
CN110672617A (en) Method for detecting defects of silk-screen area of glass cover plate of smart phone based on machine vision
CN113554582B (en) Defect detection method, device and system for functional hole in electronic equipment cover plate
CN110706224B (en) Optical element weak scratch detection method, system and device based on dark field image
CN110473201A (en) A kind of automatic testing method and device of disc surface defect
CN116503388B (en) Defect detection method, device and storage medium
CN117974601B (en) Method and system for detecting surface defects of silicon wafer based on template matching
CN108537286A (en) A kind of accurate recognition methods of complex target based on key area detection
CN110738106A (en) optical remote sensing image ship detection method based on FPGA
CN112669272B (en) AOI rapid detection method and rapid detection system
CN114283332B (en) Fuzzy clustering remote sensing image segmentation method, system, terminal and storage medium
US20240087105A1 (en) Systems and Methods for Paint Defect Detection Using Machine Learning
CN113538603B (en) Optical detection method and system based on array product and readable storage medium
CN116385430A (en) Machine vision flaw detection method, device, medium and equipment
CN112381765A (en) Equipment detection method, device, equipment and storage medium based on artificial intelligence
CN114998217A (en) Method for determining defect grade of glass substrate, computer device and storage medium
CN113808080B (en) Method for detecting number of interference fringes of glass panel of camera hole of mobile phone
CN116912475B (en) Display screen foreign matter detection method and device, electronic equipment and storage medium
CN117635687A (en) Extraction method of line structure light stripe center
CN116797550A (en) Defect detection method, device, electronic equipment and storage medium
CN110298825B (en) Method for detecting black spot defect of light guide plate
CN112818980A (en) Steel ladle number detection and identification method based on Yolov3 algorithm
Chiu et al. Effective image models for inspecting profile flaws of car mirrors with applications

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant