CN110675399A - Screen appearance flaw detection method and equipment - Google Patents
Screen appearance flaw detection method and equipment Download PDFInfo
- Publication number
- CN110675399A CN110675399A CN201911032898.XA CN201911032898A CN110675399A CN 110675399 A CN110675399 A CN 110675399A CN 201911032898 A CN201911032898 A CN 201911032898A CN 110675399 A CN110675399 A CN 110675399A
- Authority
- CN
- China
- Prior art keywords
- screen
- defect
- electronic device
- network
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Analytical Chemistry (AREA)
- Biophysics (AREA)
- Chemical & Material Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biochemistry (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention aims to provide a method and equipment for detecting screen appearance flaws, wherein an appearance image of electronic equipment is acquired; extracting a screen appearance area image of the electronic equipment from an appearance image of the electronic equipment; inputting the screen appearance area image into a model of combining the FPN network with the backbone network after training is finished; receiving a defect detection result of a screen appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect type of the screen of the electronic equipment, the position of the defect in the screen of the electronic equipment and the confidence coefficient of the defect detection result can accurately identify the defect difference of the screen appearance of the second-hand electronic equipment such as a mobile phone.
Description
Technical Field
The invention relates to the field of computers, in particular to a method and equipment for detecting screen appearance flaws.
Background
Since the conventional image processing method is largely dependent on the selection of the threshold, and the screen appearance of the handheld electronic device such as a mobile phone has different degrees of differences in aspects such as color, appearance, aging degree and the like, it is difficult to provide a certain threshold, so that the conventional image processing method is not suitable for the detection of the defects in the screen appearance.
Disclosure of Invention
The invention aims to provide a method and equipment for detecting screen appearance defects.
According to an aspect of the present invention, there is provided a screen appearance flaw detection method, including:
acquiring an appearance image of the electronic equipment;
extracting a screen appearance area image of the electronic equipment from an appearance image of the electronic equipment;
inputting the screen appearance area image into a model of combining the FPN network with the backbone network after training is finished;
receiving a defect detection result of a screen appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect type of the screen of the electronic device, the position of the defect in the screen of the electronic device, and the confidence of the defect detection result.
Further, in the above method, extracting a screen appearance region image of the electronic device from the appearance image of the electronic device includes:
and extracting the screen appearance area image of the electronic equipment from the appearance image of the electronic equipment by adopting a Unet instance segmentation mode.
Further, in the above method, the front 2 layer of the backhaul network adopts res structure, and the back 2 layer of the network adopts an initiation structure.
Further, in the above method, after receiving the defect detection result of the screen appearance area of the electronic device from the model of the FPN network in combination with the backbone network, the method further includes:
identifying whether a confidence level of the flaw detection result is greater than a first preset threshold,
and if the current value is larger than the first preset threshold value, outputting result information including the defect type of the screen of the electronic equipment and the position of the defect in the screen of the electronic equipment.
Further, before inputting the screen appearance region image into the model combining the FPN network and the backbone network, the method further includes:
presetting a model of combining an FPN network with a backbone network and initial model parameters thereof;
inputting the screen appearance area image of the sample electronic equipment into a FPN network with current model parameters and combining with a backsbone network model to obtain a defect prediction result of the screen of the sample electronic equipment, wherein the defect prediction result comprises the following steps: the defect type of the screen of the sample electronic device, the position of the defect in the screen of the sample electronic device, and the confidence of the defect detection result;
calculating a difference between the flaw prediction result and a true flaw result of the sample electronic device based on a preset objective function, identifying whether the difference is greater than a second preset threshold,
if the difference value is larger than a second preset threshold value, a fourth step of executing from the second step again after updating the model parameters of the FPN network combined with the backbone network based on the difference value;
and if the difference is smaller than or equal to a second preset threshold, step five, taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
According to another aspect of the present invention, there is also provided a screen appearance flaw detection apparatus including:
the device comprises a first device, a second device and a third device, wherein the first device is used for acquiring an appearance image of the electronic equipment;
second means for extracting a screen appearance region image of the electronic device from an appearance image of the electronic device;
the third device is used for inputting the screen appearance area image into a model of the FPN network combined with the backbone network after training is finished;
fourth means for receiving an output defect detection result of a screen appearance area of an electronic device from a model of the FPN network in combination with a backbone network, wherein the defect detection result includes: the defect type of the screen of the electronic device, the position of the defect in the screen of the electronic device, and the confidence of the defect detection result.
Further, in the foregoing device, the second means is configured to extract the screen appearance region image of the electronic device from the appearance image of the electronic device by using a Unet instance segmentation method.
Further, in the above device, the front 2 layer of the backhaul network adopts a res structure, and the rear 2 layer of the network adopts an initiation structure.
Further, in the foregoing apparatus, the fourth device is further configured to identify whether a confidence of the defect detection result is greater than a first preset threshold, and if the confidence is greater than the first preset threshold, output result information including a defect type of a screen of the electronic apparatus and a position of the defect in the screen of the electronic apparatus.
Further, the above apparatus further includes a fifth device, including:
a fifth device, configured to preset a model of the FPN network combined with the backbone network and initial model parameters thereof;
a fifth second device, configured to input the screen appearance area image of the sample electronic device into a model combining the FPN network with current model parameters and the backbone network, to obtain a defect prediction result of the screen of the sample electronic device, where the defect prediction result includes: the defect type of the screen of the sample electronic device, the position of the defect in the screen of the sample electronic device, and the confidence of the defect detection result;
a fifth third means for calculating a difference between the defect prediction result and a true defect result of the sample electronic device based on a preset objective function, and identifying whether the difference is greater than a second preset threshold, if the difference is greater than the second preset threshold, executing a fifth fourth means for restarting execution from the fifth second means after updating the model parameters of the FPN network in combination with the backbone network based on the difference;
and if the difference is smaller than or equal to a second preset threshold, executing a fifth device, and taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
The present invention also provides a computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
step S1, acquiring an appearance image of the electronic equipment;
step S2, extracting the screen appearance area image of the electronic equipment from the appearance image of the electronic equipment;
step S3, inputting the screen appearance area image into a model of the FPN network combined with the backbone network after training is finished;
step S4, receiving a defect detection result of the screen appearance area of the electronic device from the model of the FPN network combined with the backbone network, where the defect detection result includes: the defect type of the screen of the electronic device, the position of the defect in the screen of the electronic device, and the confidence of the defect detection result.
The present invention also provides a computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
step S1, acquiring an appearance image of the electronic equipment;
step S2, extracting the screen appearance area image of the electronic equipment from the appearance image of the electronic equipment;
step S3, inputting the screen appearance area image into a model of the FPN network combined with the backbone network after training is finished;
step S4, receiving a defect detection result of the screen appearance area of the electronic device from the model of the FPN network combined with the backbone network, where the defect detection result includes: the defect type of the screen of the electronic device, the position of the defect in the screen of the electronic device, and the confidence of the defect detection result.
Compared with the prior art, the method has the advantages that the appearance image of the electronic equipment is obtained; extracting a screen appearance area image of the electronic equipment from an appearance image of the electronic equipment; inputting the screen appearance area image into a model of combining the FPN network with the backbone network after training is finished; receiving a defect detection result of a screen appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect type of the screen of the electronic equipment, the position of the defect in the screen of the electronic equipment and the confidence coefficient of the defect detection result can accurately identify the defect difference of the screen appearance of the second-hand electronic equipment such as a mobile phone.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
FIG. 1 is a flow chart illustrating a method for detecting defects on a screen according to an embodiment of the invention;
FIG. 2 is a diagram illustrating a defect detection result according to an embodiment of the invention;
fig. 3 is a schematic diagram illustrating a model of a FPN network combined with a backbone network according to an embodiment of the present invention.
The same or similar reference numbers in the drawings identify the same or similar elements.
Detailed Description
The present invention is described in further detail below with reference to the attached drawing figures.
In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
The invention provides a method for detecting screen appearance flaws, which comprises the following steps:
step S1, acquiring an appearance image of the electronic equipment;
step S2, extracting the screen appearance area image of the electronic equipment from the appearance image of the electronic equipment;
step S3, inputting the screen appearance area image into a model of the FPN network combined with the backbone network after training is finished;
step S4, receiving a defect detection result of the screen appearance area of the electronic device from the model of the FPN network combined with the backbone network, where the defect detection result includes: the defect type of the screen of the electronic device, the position of the defect in the screen of the electronic device, and the confidence of the defect detection result.
The model of the FPN network in combination with the backbone network can be as shown in fig. 3.
Here, the defect detection results of the screen appearance region of the electronic device, which are received and output from the model of the FPN network combined with the backbone network, as shown in fig. 2, each defect detection result includes cls, x1, y1, x2, y2, score, where cls is a defect type, x1, y1, x2, and y2 are 4 coordinates of a position where a defect is located in the screen appearance region image, and score is a confidence of the defect.
The invention mainly utilizes the improved characteristic pyramid (FPN) network and the deep learning model of the backbone network to accurately identify the defect difference of the screen appearance of the second-hand electronic equipment such as a mobile phone.
In an embodiment of the method for detecting the screen appearance flaws, in step S2, the method for extracting the screen appearance area image of the electronic device from the appearance image of the electronic device includes:
and extracting the screen appearance area image of the electronic equipment from the appearance image of the electronic equipment by adopting a Unet instance segmentation mode.
Here, the screen appearance region image can be obtained quickly and efficiently by the Unet instance division.
In an embodiment of the screen appearance flaw detection method, the front 2 layers of the backbone network adopt a res structure, and the rear 2 layers of the network adopt an acceptance structure.
In an embodiment of the method for detecting the defect of the screen appearance of the present invention, after the step S4, receiving the defect detection result of the screen appearance area of the electronic device from the model of the FPN network combined with the backbone network, the method further includes:
identifying whether a confidence level of the flaw detection result is greater than a first preset threshold,
and if the current value is larger than the first preset threshold value, outputting result information including the defect type of the screen of the electronic equipment and the position of the defect in the screen of the electronic equipment.
Here, the defect kinds of the screen may sequentially include a shallow scratch, a hard scratch, and a chipping kind, of which grades are sequentially increased.
In this embodiment, by identifying the confidence of the flaw detection result, a reliable result can be screened from the flaw detection result and output.
In an embodiment of the method for detecting the screen appearance flaws, before the step S3 of inputting the screen appearance region image into a model of FPN network combined with backbone network, the method further includes:
presetting a model of combining an FPN network with a backbone network and initial model parameters thereof;
inputting the screen appearance area image of the sample electronic equipment into a FPN network with current model parameters and combining with a backsbone network model to obtain a defect prediction result of the screen of the sample electronic equipment, wherein the defect prediction result comprises the following steps: the defect type of the screen of the sample electronic device, the position of the defect in the screen of the sample electronic device, and the confidence of the defect detection result;
calculating a difference between the flaw prediction result and a true flaw result of the sample electronic device based on a preset objective function, identifying whether the difference is greater than a second preset threshold,
if the difference value is larger than a second preset threshold value, a fourth step of executing from the second step again after updating the model parameters of the FPN network combined with the backbone network based on the difference value;
and if the difference is smaller than or equal to a second preset threshold, step five, taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
And circularly training the model of the FPN network combined with the backbone network by identifying whether the difference value is greater than a second preset threshold, so as to obtain a reliable model.
The invention provides a screen appearance flaw detection device, comprising:
the device comprises a first device, a second device and a third device, wherein the first device is used for acquiring an appearance image of the electronic equipment;
second means for extracting a screen appearance region image of the electronic device from an appearance image of the electronic device;
the third device is used for inputting the screen appearance area image into a model of the FPN network combined with the backbone network after training is finished;
fourth means for receiving an output defect detection result of a screen appearance area of an electronic device from a model of the FPN network in combination with a backbone network, wherein the defect detection result includes: the defect type of the screen of the electronic device, the position of the defect in the screen of the electronic device, and the confidence of the defect detection result.
Here, the fault detection results of the screen appearance region of the electronic device, which are output from the model of the FPN network combined with the backbone network, each fault detection result includes cls, x1, y1, x2, y2, and score, where cls is a defect type, x1, y1, x2, and y2 are 4 coordinates of a position where a fault is located in the screen appearance region image, and score is a confidence of the fault.
The invention mainly utilizes the improved characteristic pyramid (FPN) network and the deep learning model of the backbone network to accurately identify the screen appearance difference of the second-hand electronic equipment such as a mobile phone.
In an embodiment of the method for detecting the screen appearance flaws, the second device is configured to extract a screen appearance area image of the electronic device from an appearance image of the electronic device in a manner of using a pnet instance segmentation.
Here, the screen appearance region image can be obtained quickly and efficiently by the Unet instance division.
In an embodiment of the screen appearance flaw detection method, the front 2 layers of the backbone network adopt a res structure, and the rear 2 layers of the network adopt an acceptance structure.
In an embodiment of the method for detecting the screen appearance defect, the fourth device is further configured to identify whether a confidence of the defect detection result is greater than a first preset threshold, and if the confidence is greater than the first preset threshold, output result information including a defect type of a screen of the electronic device and a position of the defect in the screen of the electronic device.
Here, the defect kinds of the screen may sequentially include a shallow scratch, a hard scratch, and a chipping kind, of which grades are sequentially increased.
In this embodiment, by identifying the confidence of the flaw detection result, a reliable result can be screened from the flaw detection result and output.
In an embodiment of the method for detecting the screen appearance defect of the present invention, the method further includes a fifth apparatus, including:
a fifth device, configured to preset a model of the FPN network combined with the backbone network and initial model parameters thereof;
a fifth second device, configured to input the screen appearance area image of the sample electronic device into a model combining the FPN network with current model parameters and the backbone network, to obtain a defect prediction result of the screen of the sample electronic device, where the defect prediction result includes: the defect type of the screen of the sample electronic device, the position of the defect in the screen of the sample electronic device, and the confidence of the defect detection result;
a fifth third means for calculating a difference between the defect prediction result and a true defect result of the sample electronic device based on a preset objective function, and identifying whether the difference is greater than a second preset threshold, if the difference is greater than the second preset threshold, executing a fifth fourth means for restarting execution from the fifth second means after updating the model parameters of the FPN network in combination with the backbone network based on the difference;
and if the difference is smaller than or equal to a second preset threshold, executing a fifth device, and taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
And circularly training the model of the FPN network combined with the backbone network by identifying whether the difference value is greater than a second preset threshold, so as to obtain a reliable model.
The present invention also provides a computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
step S1, acquiring an appearance image of the electronic equipment;
step S2, extracting the screen appearance area image of the electronic equipment from the appearance image of the electronic equipment;
step S3, inputting the screen appearance area image into a model of the FPN network combined with the backbone network after training is finished;
step S4, receiving a defect detection result of the screen appearance area of the electronic device from the model of the FPN network combined with the backbone network, where the defect detection result includes: the defect type of the screen of the electronic device, the position of the defect in the screen of the electronic device, and the confidence of the defect detection result.
The present invention also provides a computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
step S1, acquiring an appearance image of the electronic equipment;
step S2, extracting the screen appearance area image of the electronic equipment from the appearance image of the electronic equipment;
step S3, inputting the screen appearance area image into a model of the FPN network combined with the backbone network after training is finished;
step S4, receiving a defect detection result of the screen appearance area of the electronic device from the model of the FPN network combined with the backbone network, where the defect detection result includes: the defect type of the screen of the electronic device, the position of the defect in the screen of the electronic device, and the confidence of the defect detection result.
For details of embodiments of each device and storage medium of the present invention, reference may be made to corresponding parts of each method embodiment, and details are not described herein again.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
It should be noted that the present invention may be implemented in software and/or in a combination of software and hardware, for example, as an Application Specific Integrated Circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention may be executed by a processor to implement the steps or functions described above. Also, the software programs (including associated data structures) of the present invention can be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Further, some of the steps or functions of the present invention may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present invention can be applied as a computer program product, such as computer program instructions, which when executed by a computer, can invoke or provide the method and/or technical solution according to the present invention through the operation of the computer. Program instructions which invoke the methods of the present invention may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the invention herein comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or solution according to embodiments of the invention as described above.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (12)
1. A screen appearance flaw detection method, wherein the method comprises the following steps:
acquiring an appearance image of the electronic equipment;
extracting a screen appearance area image of the electronic equipment from an appearance image of the electronic equipment;
inputting the screen appearance area image into a model of combining the FPN network with the backbone network after training is finished;
receiving a defect detection result of a screen appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect type of the screen of the electronic device, the position of the defect in the screen of the electronic device, and the confidence of the defect detection result.
2. The method of claim 1, wherein extracting the screen appearance region image of the electronic device from the appearance image of the electronic device comprises:
and extracting the screen appearance area image of the electronic equipment from the appearance image of the electronic equipment by adopting a Unet instance segmentation mode.
3. The method of claim 1, wherein the first 2 layers of the backhaul network adopt res structure, and the last 2 layers of the network adopt initiation structure.
4. The method of claim 1, wherein after receiving the output defect detection result of the screen appearance area of the electronic device from the model of the FPN network in combination with the backbone network, further comprising:
identifying whether a confidence level of the flaw detection result is greater than a first preset threshold,
and if the current value is larger than the first preset threshold value, outputting result information including the defect type of the screen of the electronic equipment and the position of the defect in the screen of the electronic equipment.
5. The method of claim 1, wherein before inputting the screen appearance region image into the model of the FPN network in combination with the backbone network, further comprising:
presetting a model of combining an FPN network with a backbone network and initial model parameters thereof;
inputting the screen appearance area image of the sample electronic equipment into a FPN network with current model parameters and combining with a backsbone network model to obtain a defect prediction result of the screen of the sample electronic equipment, wherein the defect prediction result comprises the following steps: the defect type of the screen of the sample electronic device, the position of the defect in the screen of the sample electronic device, and the confidence of the defect detection result;
calculating a difference between the flaw prediction result and a true flaw result of the sample electronic device based on a preset objective function, identifying whether the difference is greater than a second preset threshold,
if the difference value is larger than a second preset threshold value, a fourth step of executing from the second step again after updating the model parameters of the FPN network combined with the backbone network based on the difference value;
and if the difference is smaller than or equal to a second preset threshold, step five, taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
6. A screen appearance flaw detection apparatus, wherein the apparatus comprises:
the device comprises a first device, a second device and a third device, wherein the first device is used for acquiring an appearance image of the electronic equipment;
second means for extracting a screen appearance region image of the electronic device from an appearance image of the electronic device;
the third device is used for inputting the screen appearance area image into a model of the FPN network combined with the backbone network after training is finished;
fourth means for receiving an output defect detection result of a screen appearance area of an electronic device from a model of the FPN network in combination with a backbone network, wherein the defect detection result includes: the defect type of the screen of the electronic device, the position of the defect in the screen of the electronic device, and the confidence of the defect detection result.
7. The apparatus of claim 6, wherein the second means is configured to extract the screen appearance region image of the electronic device from the appearance image of the electronic device by means of Unet instance segmentation.
8. The apparatus of claim 6, wherein the front 2 layers of the backhaul network adopt res structure, and the back 2 layers of the network adopt initiation structure.
9. The apparatus of claim 6, wherein the fourth means is further configured to identify whether a confidence of the defect detection result is greater than a first preset threshold, and if so, output result information including a defect type of a screen of the electronic device and a position of the defect in the screen of the electronic device.
10. The apparatus of claim 6, further comprising a fifth apparatus comprising:
a fifth device, configured to preset a model of the FPN network combined with the backbone network and initial model parameters thereof;
a fifth second device, configured to input the screen appearance area image of the sample electronic device into a model combining the FPN network with current model parameters and the backbone network, to obtain a defect prediction result of the screen of the sample electronic device, where the defect prediction result includes: the defect type of the screen of the sample electronic device, the position of the defect in the screen of the sample electronic device, and the confidence of the defect detection result;
a fifth third means for calculating a difference between the defect prediction result and a true defect result of the sample electronic device based on a preset objective function, and identifying whether the difference is greater than a second preset threshold, if the difference is greater than the second preset threshold, executing a fifth fourth means for restarting execution from the fifth second means after updating the model parameters of the FPN network in combination with the backbone network based on the difference;
and if the difference is smaller than or equal to a second preset threshold, executing a fifth device, and taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
11. A computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring an appearance image of the electronic equipment;
extracting a screen appearance area image of the electronic equipment from an appearance image of the electronic equipment;
inputting the screen appearance area image into a model of combining the FPN network with the backbone network after training is finished;
receiving a defect detection result of a screen appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect type of the screen of the electronic device, the position of the defect in the screen of the electronic device, and the confidence of the defect detection result.
12. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
acquiring an appearance image of the electronic equipment;
extracting a screen appearance area image of the electronic equipment from an appearance image of the electronic equipment;
inputting the screen appearance area image into a model of combining the FPN network with the backbone network after training is finished;
receiving a defect detection result of a screen appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect type of the screen of the electronic device, the position of the defect in the screen of the electronic device, and the confidence of the defect detection result.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911032898.XA CN110675399A (en) | 2019-10-28 | 2019-10-28 | Screen appearance flaw detection method and equipment |
| PCT/CN2020/120874 WO2021082918A1 (en) | 2019-10-28 | 2020-10-14 | Screen appearance defect detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911032898.XA CN110675399A (en) | 2019-10-28 | 2019-10-28 | Screen appearance flaw detection method and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110675399A true CN110675399A (en) | 2020-01-10 |
Family
ID=69084568
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911032898.XA Pending CN110675399A (en) | 2019-10-28 | 2019-10-28 | Screen appearance flaw detection method and equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110675399A (en) |
| WO (1) | WO2021082918A1 (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111311556A (en) * | 2020-01-21 | 2020-06-19 | 上海悦易网络信息技术有限公司 | Mobile phone defect position identification method and equipment |
| CN111325717A (en) * | 2020-01-21 | 2020-06-23 | 上海悦易网络信息技术有限公司 | Mobile phone defect position identification method and equipment |
| CN111325716A (en) * | 2020-01-21 | 2020-06-23 | 上海悦易网络信息技术有限公司 | Screen scratch fragmentation detection method and equipment |
| CN111539456A (en) * | 2020-04-02 | 2020-08-14 | 浙江华睿科技有限公司 | Target identification method and device |
| WO2021082918A1 (en) * | 2019-10-28 | 2021-05-06 | 上海悦易网络信息技术有限公司 | Screen appearance defect detection method and device |
| WO2021082921A1 (en) * | 2019-10-28 | 2021-05-06 | 上海悦易网络信息技术有限公司 | Back cover appearance defect detection method for electronic apparatus, and apparatus |
| CN113052798A (en) * | 2021-03-08 | 2021-06-29 | 广州绿怡信息科技有限公司 | Screen aging detection model training method and screen aging detection method |
| CN114885056A (en) * | 2022-05-06 | 2022-08-09 | 深圳市金得源科技有限公司 | Mobile terminal detection method, system, detection device and storage medium |
| US11798250B2 (en) | 2019-02-18 | 2023-10-24 | Ecoatm, Llc | Neural network based physical condition evaluation of electronic devices, and associated systems and methods |
| US11843206B2 (en) | 2019-02-12 | 2023-12-12 | Ecoatm, Llc | Connector carrier for electronic device kiosk |
| US11922467B2 (en) | 2020-08-17 | 2024-03-05 | ecoATM, Inc. | Evaluating an electronic device using optical character recognition |
| US11989710B2 (en) | 2018-12-19 | 2024-05-21 | Ecoatm, Llc | Systems and methods for vending and/or purchasing mobile phones and other electronic devices |
| US12033454B2 (en) | 2020-08-17 | 2024-07-09 | Ecoatm, Llc | Kiosk for evaluating and purchasing used electronic devices |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11900581B2 (en) | 2020-09-22 | 2024-02-13 | Future Dial, Inc. | Cosmetic inspection system |
| US11836912B2 (en) * | 2020-09-22 | 2023-12-05 | Future Dial, Inc. | Grading cosmetic appearance of a test object based on multi-region determination of cosmetic defects |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106875381A (en) * | 2017-01-17 | 2017-06-20 | 同济大学 | A kind of phone housing defect inspection method based on deep learning |
| US20180300880A1 (en) * | 2017-04-12 | 2018-10-18 | Here Global B.V. | Small object detection from a large image |
| CN109509172A (en) * | 2018-09-25 | 2019-03-22 | 无锡动视宫原科技有限公司 | A kind of liquid crystal display flaw detection method and system based on deep learning |
| CN109784181A (en) * | 2018-12-14 | 2019-05-21 | 平安科技（深圳）有限公司 | Picture watermark recognition methods, device, equipment and computer readable storage medium |
| CN109859190A (en) * | 2019-01-31 | 2019-06-07 | 北京工业大学 | A kind of target area detection method based on deep learning |
| CN110378420A (en) * | 2019-07-19 | 2019-10-25 | Oppo广东移动通信有限公司 | A kind of image detecting method, device and computer readable storage medium |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8068425B2 (en) * | 2008-04-09 | 2011-11-29 | Embarq Holdings Company, Llc | System and method for using network performance information to determine improved measures of path states |
| CN108846841A (en) * | 2018-07-02 | 2018-11-20 | 北京百度网讯科技有限公司 | Display screen quality determining method, device, electronic equipment and storage medium |
| CN109461149A (en) * | 2018-10-31 | 2019-03-12 | 泰州市创新电子有限公司 | The intelligent checking system and method for lacquered surface defect |
| CN109711474B (en) * | 2018-12-24 | 2023-01-17 | 中山大学 | Aluminum product surface defect detection algorithm based on deep learning |
| CN109886077B (en) * | 2018-12-28 | 2021-07-09 | 北京旷视科技有限公司 | Image recognition method and device, computer equipment and storage medium |
| CN110110661A (en) * | 2019-05-07 | 2019-08-09 | 西南石油大学 | A kind of rock image porosity type recognition methods based on unet segmentation |
| CN110827249A (en) * | 2019-10-28 | 2020-02-21 | 上海悦易网络信息技术有限公司 | Electronic equipment backboard appearance flaw detection method and equipment |
| CN110675399A (en) * | 2019-10-28 | 2020-01-10 | 上海悦易网络信息技术有限公司 | Screen appearance flaw detection method and equipment |
| CN110827246A (en) * | 2019-10-28 | 2020-02-21 | 上海悦易网络信息技术有限公司 | Electronic equipment frame appearance flaw detection method and equipment |
| CN110827244A (en) * | 2019-10-28 | 2020-02-21 | 上海悦易网络信息技术有限公司 | Method and equipment for detecting appearance flaws of electronic equipment |
| CN110796646A (en) * | 2019-10-28 | 2020-02-14 | 上海悦易网络信息技术有限公司 | Method and device for detecting defects of screen area of electronic device |
| CN110796647A (en) * | 2019-10-28 | 2020-02-14 | 上海悦易网络信息技术有限公司 | Method and device for detecting defects of screen area of electronic device |
-
2019
- 2019-10-28 CN CN201911032898.XA patent/CN110675399A/en active Pending
-
2020
- 2020-10-14 WO PCT/CN2020/120874 patent/WO2021082918A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106875381A (en) * | 2017-01-17 | 2017-06-20 | 同济大学 | A kind of phone housing defect inspection method based on deep learning |
| US20180300880A1 (en) * | 2017-04-12 | 2018-10-18 | Here Global B.V. | Small object detection from a large image |
| CN109509172A (en) * | 2018-09-25 | 2019-03-22 | 无锡动视宫原科技有限公司 | A kind of liquid crystal display flaw detection method and system based on deep learning |
| CN109784181A (en) * | 2018-12-14 | 2019-05-21 | 平安科技（深圳）有限公司 | Picture watermark recognition methods, device, equipment and computer readable storage medium |
| CN109859190A (en) * | 2019-01-31 | 2019-06-07 | 北京工业大学 | A kind of target area detection method based on deep learning |
| CN110378420A (en) * | 2019-07-19 | 2019-10-25 | Oppo广东移动通信有限公司 | A kind of image detecting method, device and computer readable storage medium |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11989710B2 (en) | 2018-12-19 | 2024-05-21 | Ecoatm, Llc | Systems and methods for vending and/or purchasing mobile phones and other electronic devices |
| US11843206B2 (en) | 2019-02-12 | 2023-12-12 | Ecoatm, Llc | Connector carrier for electronic device kiosk |
| US11798250B2 (en) | 2019-02-18 | 2023-10-24 | Ecoatm, Llc | Neural network based physical condition evaluation of electronic devices, and associated systems and methods |
| WO2021082918A1 (en) * | 2019-10-28 | 2021-05-06 | 上海悦易网络信息技术有限公司 | Screen appearance defect detection method and device |
| WO2021082921A1 (en) * | 2019-10-28 | 2021-05-06 | 上海悦易网络信息技术有限公司 | Back cover appearance defect detection method for electronic apparatus, and apparatus |
| CN111325716A (en) * | 2020-01-21 | 2020-06-23 | 上海悦易网络信息技术有限公司 | Screen scratch fragmentation detection method and equipment |
| CN111325717A (en) * | 2020-01-21 | 2020-06-23 | 上海悦易网络信息技术有限公司 | Mobile phone defect position identification method and equipment |
| CN111311556A (en) * | 2020-01-21 | 2020-06-19 | 上海悦易网络信息技术有限公司 | Mobile phone defect position identification method and equipment |
| CN111311556B (en) * | 2020-01-21 | 2023-02-03 | 上海万物新生环保科技集团有限公司 | Mobile phone defect position identification method and equipment |
| CN111325717B (en) * | 2020-01-21 | 2023-08-29 | 上海万物新生环保科技集团有限公司 | Mobile phone defect position identification method and equipment |
| CN111325716B (en) * | 2020-01-21 | 2023-09-01 | 上海万物新生环保科技集团有限公司 | Screen scratch and fragmentation detection method and equipment |
| CN111539456B (en) * | 2020-04-02 | 2024-03-01 | 浙江华睿科技股份有限公司 | Target identification method and device |
| CN111539456A (en) * | 2020-04-02 | 2020-08-14 | 浙江华睿科技有限公司 | Target identification method and device |
| US11922467B2 (en) | 2020-08-17 | 2024-03-05 | ecoATM, Inc. | Evaluating an electronic device using optical character recognition |
| US12033454B2 (en) | 2020-08-17 | 2024-07-09 | Ecoatm, Llc | Kiosk for evaluating and purchasing used electronic devices |
| CN113052798A (en) * | 2021-03-08 | 2021-06-29 | 广州绿怡信息科技有限公司 | Screen aging detection model training method and screen aging detection method |
| CN114885056A (en) * | 2022-05-06 | 2022-08-09 | 深圳市金得源科技有限公司 | Mobile terminal detection method, system, detection device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021082918A1 (en) | 2021-05-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110675399A (en) | Screen appearance flaw detection method and equipment | |
| CN110827249A (en) | Electronic equipment backboard appearance flaw detection method and equipment | |
| CN110827246A (en) | Electronic equipment frame appearance flaw detection method and equipment | |
| CN110796646A (en) | Method and device for detecting defects of screen area of electronic device | |
| CN110796647A (en) | Method and device for detecting defects of screen area of electronic device | |
| CN110059468B (en) | Applet risk identification method and device | |
| CN110175609B (en) | Interface element detection method, device and equipment | |
| CN107229559B (en) | Detection method and device for testing integrity of service system | |
| CN111175318A (en) | Screen scratch fragmentation detection method and equipment | |
| CN110348392B (en) | Vehicle matching method and device | |
| CN111210473A (en) | Mobile phone contour positioning method and equipment | |
| CN109978044B (en) | Training data generation method and device, and model training method and device | |
| CN110708568B (en) | Video content mutation detection method and device | |
| US20230401691A1 (en) | Image defect detection method, electronic device and readable storage medium | |
| CN111401238A (en) | Method and device for detecting character close-up segments in video | |
| US11728914B2 (en) | Detection device, detection method, and program | |
| CN110728193B (en) | Method and device for detecting richness characteristics of face image | |
| CN110134568A (en) | A kind of application program launching time detection method, device and equipment | |
| CN117333383B (en) | Surface defect detection method, device and equipment | |
| CN113466839A (en) | Side-scan sonar sea bottom line detection method and device | |
| CN112633515A (en) | Model training method and device based on sample elimination | |
| CN112055258B (en) | Time delay testing method and device for loading live broadcast picture, electronic equipment and storage medium | |
| US20220122341A1 (en) | Target detection method and apparatus, electronic device, and computer storage medium | |
| CN111327745B (en) | Method and equipment for testing silencing key of mobile terminal | |
| CN112801987B (en) | Mobile phone part abnormity detection method and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 1101-1103, No. 433, Songhu Road, Yangpu District, Shanghai Applicant after: Shanghai wanwansheng Environmental Protection Technology Group Co.,Ltd. Address before: Room 1101-1103, No. 433, Songhu Road, Yangpu District, Shanghai Applicant before: SHANGHAI YUEYI NETWORK INFORMATION TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200110 |
|
| RJ01 | Rejection of invention patent application after publication |