CN110827246A - Electronic equipment frame appearance flaw detection method and equipment - Google Patents
Electronic equipment frame appearance flaw detection method and equipment Download PDFInfo
- Publication number
- CN110827246A CN110827246A CN201911032912.6A CN201911032912A CN110827246A CN 110827246 A CN110827246 A CN 110827246A CN 201911032912 A CN201911032912 A CN 201911032912A CN 110827246 A CN110827246 A CN 110827246A
- Authority
- CN
- China
- Prior art keywords
- frame
- image
- electronic equipment
- appearance
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention aims to provide a method and equipment for detecting frame appearance flaws of electronic equipment, wherein an appearance image of the electronic equipment is obtained; extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the frame appearance area image into an image with the same length and width; inputting the adjusted frame appearance area image into a model of combining the FPN network and the backbone network after training is finished; receiving a defect detection result of a frame appearance area of the electronic device, which is output from a model combining the FPN network and the backbone network, wherein the defect detection result comprises: the defect type of the frame of the electronic equipment, the position of the defect in the frame of the electronic equipment and the confidence coefficient of the defect detection result can accurately identify the defect difference of the appearance of the frame of the electronic equipment of the second-hand electronic equipment such as a mobile phone.
Description
Technical Field
The invention relates to the field of computers, in particular to a method and equipment for detecting appearance flaws of a frame of electronic equipment.
Background
At present, the defect detection of the appearance of the frame of the electronic equipment such as a mobile phone and the like in the second-hand electronic equipment is mainly based on the traditional image algorithm and is carried out by the modes of color space transformation, filtering, feature point extraction and mode matching, and the defect of a certain area can only be detected based on the traditional detection method, but the definition of the defect cannot be distinguished.
Disclosure of Invention
The invention aims to provide a method and equipment for detecting appearance defects of a frame of electronic equipment.
According to an aspect of the present invention, a method for detecting an appearance defect of a bezel of an electronic device is provided, the method comprising:
acquiring an appearance image of the electronic equipment;
extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the frame appearance area image into an image with the same length and width;
inputting the adjusted frame appearance area image into a model of combining the FPN network and the backbone network after training is finished;
receiving a defect detection result of a frame appearance area of the electronic device, which is output from a model combining the FPN network and the backbone network, wherein the defect detection result comprises: the defect detection method comprises the steps of detecting defects of a frame of the electronic device, determining the positions of the defects in the frame of the electronic device and determining the confidence of the defect detection result.
Further, in the above method, extracting a frame appearance region image of the electronic device from the appearance image of the electronic device includes:
and extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment by adopting a Unet instance segmentation mode.
Further, in the above method, the front 2 layer of the backhaul network adopts res structure, and the back 2 layer of the network adopts an initiation structure.
Further, in the above method, after receiving the defect detection result of the bezel appearance area of the electronic device from the model of the FPN network in combination with the backbone network, the method further includes:
identifying whether a confidence level of the flaw detection result is greater than a first preset threshold,
and if the defect type is larger than the first preset threshold, outputting result information including the defect type of the frame of the electronic equipment and the position of the defect in the frame of the electronic equipment.
Further, in the above method, before inputting the frame appearance region image into a model combining the FPN network and the backbone network, the method further includes:
presetting a model of combining an FPN network with a backbone network and initial model parameters thereof;
inputting the frame appearance area image of the sample electronic equipment into a FPN network with current model parameters and combining with a backbone network model to obtain a defect prediction result of the frame of the sample electronic equipment, wherein the defect prediction result comprises the following steps: the defect type of the frame of the sample electronic equipment, the position of the defect in the frame of the sample electronic equipment and the confidence coefficient of the defect detection result;
calculating a difference between the flaw prediction result and a true flaw result of the sample electronic device based on a preset objective function, identifying whether the difference is greater than a second preset threshold,
if the difference value is larger than a second preset threshold value, a fourth step of executing from the second step again after updating the model parameters of the FPN network combined with the backbone network based on the difference value;
and if the difference is smaller than or equal to a second preset threshold, step five, taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
Further, in the foregoing method, adjusting the border appearance area image to an image with the same length and width includes:
and zooming the length direction of the frame appearance area image, and filling the width direction of the frame appearance area image.
According to another aspect of the present invention, there is also provided an electronic device bezel appearance defect detecting apparatus, including:
the device comprises a first device, a second device and a third device, wherein the first device is used for acquiring an appearance image of the electronic equipment;
the second device is used for extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment and adjusting the frame appearance area image into an image with the same length and width;
the third device is used for inputting the adjusted image of the frame appearance area into a model of the FPN network combined with the backbone network after training is finished;
a fourth device, configured to receive a defect detection result of a bezel appearance area of an electronic device from a model combining the FPN network and the backbone network, where the defect detection result includes: the defect detection method comprises the steps of detecting defects of a frame of the electronic device, determining the positions of the defects in the frame of the electronic device and determining the confidence of the defect detection result.
Further, in the foregoing device, the second means is configured to extract a frame appearance region image of the electronic device from the appearance image of the electronic device by using a Unet instance segmentation method.
Further, in the above device, the front 2 layer of the backhaul network adopts a res structure, and the rear 2 layer of the network adopts an initiation structure.
Further, in the foregoing apparatus, the fourth device is further configured to identify whether a confidence of the defect detection result is greater than a first preset threshold, and if the confidence is greater than the first preset threshold, output result information including a defect type of a frame of the electronic apparatus and a position of the defect in the frame of the electronic apparatus.
Further, the above apparatus further includes a fifth device, including:
a fifth device, configured to preset a model of the FPN network combined with the backbone network and initial model parameters thereof;
a fifth second device, configured to input the frame appearance area image of the sample electronic device into a model combining a back bone network and an FPN network with current model parameters, to obtain a defect prediction result of the frame of the sample electronic device, where the defect prediction result includes: the defect type of the frame of the sample electronic equipment, the position of the defect in the frame of the sample electronic equipment and the confidence coefficient of the defect detection result;
a fifth third means for calculating a difference between the defect prediction result and a true defect result of the sample electronic device based on a preset objective function, and identifying whether the difference is greater than a second preset threshold, if the difference is greater than the second preset threshold, executing a fifth fourth means for restarting execution from the fifth second means after updating the model parameters of the FPN network in combination with the backbone network based on the difference;
and if the difference is smaller than or equal to a second preset threshold, executing a fifth device, and taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
Further, in the foregoing device, the second means is configured to scale a length direction of the frame appearance area image and fill a width direction of the frame appearance area image.
The present invention also provides a computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring an appearance image of the electronic equipment;
extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the frame appearance area image into an image with the same length and width;
inputting the adjusted frame appearance area image into a model of combining the FPN network and the backbone network after training is finished;
receiving a defect detection result of a frame appearance area of the electronic device, which is output from a model combining the FPN network and the backbone network, wherein the defect detection result comprises: the defect detection method comprises the steps of detecting defects of a frame of the electronic device, determining the positions of the defects in the frame of the electronic device and determining the confidence of the defect detection result.
The present invention also provides a computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
acquiring an appearance image of the electronic equipment;
extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the frame appearance area image into an image with the same length and width;
inputting the adjusted frame appearance area image into a model of combining the FPN network and the backbone network after training is finished;
receiving a defect detection result of a frame appearance area of the electronic device, which is output from a model combining the FPN network and the backbone network, wherein the defect detection result comprises: the defect detection method comprises the steps of detecting defects of a frame of the electronic device, determining the positions of the defects in the frame of the electronic device and determining the confidence of the defect detection result.
Compared with the prior art, the method has the advantages that the appearance image of the electronic equipment is obtained; extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the frame appearance area image into an image with the same length and width; inputting the adjusted frame appearance area image into a model of combining the FPN network and the backbone network after training is finished; receiving a defect detection result of a frame appearance area of the electronic device, which is output from a model combining the FPN network and the backbone network, wherein the defect detection result comprises: the defect type of the frame of the electronic equipment, the position of the defect in the frame of the electronic equipment and the confidence coefficient of the defect detection result can accurately identify the defect difference of the appearance of the frame of the electronic equipment of the second-hand electronic equipment such as a mobile phone.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
FIG. 1 is a flowchart illustrating a method for detecting appearance defects of a bezel of an electronic device according to an embodiment of the invention;
FIG. 2 is a diagram illustrating a defect detection result according to an embodiment of the invention;
fig. 3 is a schematic diagram illustrating a model of a FPN network combined with a backbone network according to an embodiment of the present invention.
The same or similar reference numbers in the drawings identify the same or similar elements.
Detailed Description
The present invention is described in further detail below with reference to the attached drawing figures.
In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
The invention provides a method for detecting appearance flaws of a frame of electronic equipment, which comprises the following steps:
step S1, acquiring an appearance image of the electronic equipment;
step S2, extracting a frame appearance area image of the electronic device from the appearance image of the electronic device, and adjusting the frame appearance area image to an image with the same length and width;
here, the bezel appearance area of the electronic device includes a side area, in which components such as an earphone hole, a speaker, a charging hole, and the like are generally installed, in addition to the front screen area and the rear area where the electronics are disposed.
The frame appearance area image has the condition of abnormal aspect ratio, so that subsequent model identification is facilitated, image loss is avoided, and the aspect ratio of the frame appearance area image needs to be adjusted to be 1: 1; step S3, inputting the adjusted frame appearance area image into a model of the FPN network combined with the backbone network after training is finished;
step S4, receiving and outputting a defect detection result of the frame appearance area of the electronic device from the model of the FPN network combined with the backbone network, where the defect detection result includes: the defect detection method comprises the steps of detecting defects of a frame of the electronic device, determining the positions of the defects in the frame of the electronic device and determining the confidence of the defect detection result.
The model of the FPN network in combination with the backbone network can be as shown in fig. 3.
Here, the defect detection results of the frame appearance area of the electronic device, which are received and output from the model of the FPN network combined with the backbone network, as shown in fig. 2, each defect detection result includes cls, x1, y1, x2, y2, score, where cls is a defect type, x1, y1, x2, and y2 are 4 coordinates of a position where a defect is located in the frame appearance area image, and score is a confidence of the defect.
The method mainly utilizes the improved characteristic pyramid (FPN) network and the deep learning model of the backbone network, can accurately identify the defect difference of the frame appearance of the electronic equipment of the second-hand electronic equipment such as a mobile phone, and can accurately distinguish the defect types.
In an embodiment of the method for detecting the appearance defects of the bezel of the electronic device, in step S2, the extracting the image of the appearance area of the bezel of the electronic device from the appearance image of the electronic device includes:
and extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment by adopting a Unet instance segmentation mode.
Here, the frame appearance region image can be obtained quickly and efficiently by the Unet instance division.
In an embodiment of the method for detecting the appearance flaws of the frame of the electronic device, the front 2 layers of the backbone network adopt a res structure, and the rear 2 layers of the network adopt an acceptance structure.
In an embodiment of the method for detecting defect in the appearance of the electronic device border, in step S4, after receiving the defect detection result of the border appearance area of the electronic device from the model of the FPN network combined with the backbone network, the method further includes:
identifying whether a confidence level of the flaw detection result is greater than a first preset threshold,
and if the defect type is larger than the first preset threshold, outputting result information including the defect type of the frame of the electronic equipment and the position of the defect in the frame of the electronic equipment.
Here, the defect types of the electronic device bezel may include: cracks, stent screen separation, deformation, chipping loss, large area paint drop, small area paint drop (deformation, indentation exposes color), indentation and no discoloration, deep scratch and different color from the surroundings, small dot and different color from the surroundings, chipping, and the like.
In this embodiment, by identifying the confidence of the flaw detection result, a reliable result can be screened from the flaw detection result and output.
In an embodiment of the method for detecting the frame appearance defects of the electronic device, before the step S3 of inputting the frame appearance region image into the model of the FPN network combined with the backbone network, the method further includes:
presetting a model of combining an FPN network with a backbone network and initial model parameters thereof;
inputting the frame appearance area image of the sample electronic equipment into a FPN network with current model parameters and combining with a backbone network model to obtain a defect prediction result of the frame of the sample electronic equipment, wherein the defect prediction result comprises the following steps: the defect type of the frame of the sample electronic equipment, the position of the defect in the frame of the sample electronic equipment and the confidence coefficient of the defect detection result;
calculating a difference between the flaw prediction result and a true flaw result of the sample electronic device based on a preset objective function, identifying whether the difference is greater than a second preset threshold,
if the difference value is larger than a second preset threshold value, a fourth step of executing from the second step again after updating the model parameters of the FPN network combined with the backbone network based on the difference value;
and if the difference is smaller than or equal to a second preset threshold, step five, taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
And circularly training the model of the FPN network combined with the backbone network by identifying whether the difference value is greater than a second preset threshold, so as to obtain a reliable model.
In an embodiment of the method for detecting the frame appearance flaws of the electronic device, adjusting the frame appearance area image to an image with the same length and width includes:
and zooming the length direction of the frame appearance area image, and filling the width direction of the frame appearance area image.
In the processing, the long side of the frame appearance area image is scaled, and the short side is filled, so as to obtain an image with the frame appearance area image adjusted to have the same length and width.
The invention provides an electronic equipment frame appearance flaw detection device, which comprises:
the device comprises a first device, a second device and a third device, wherein the first device is used for acquiring an appearance image of the electronic equipment;
the second device is used for extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment and adjusting the frame appearance area image into an image with the same length and width;
here, the bezel appearance area of the electronic device includes a side area, in which components such as an earphone hole, a speaker, a charging hole, and the like are generally installed, in addition to the front screen area and the rear area where the electronics are disposed.
The frame appearance area image has the condition of abnormal aspect ratio, so that subsequent model identification is facilitated, image loss is avoided, and the aspect ratio of the frame appearance area image needs to be adjusted to be 1: 1;
the third device is used for inputting the adjusted image of the frame appearance area into a model of the FPN network combined with the backbone network after training is finished;
a fourth device, configured to receive a defect detection result of a bezel appearance area of an electronic device from a model combining the FPN network and the backbone network, where the defect detection result includes: the defect detection method comprises the steps of detecting defects of a frame of the electronic device, determining the positions of the defects in the frame of the electronic device and determining the confidence of the defect detection result.
Here, the defect detection results of the frame appearance area of the electronic device, which are output from the model of the FPN network combined with the backbone network, each defect detection result includes cls, x1, y1, x2, y2, and score, where cls is a defect type, x1, y1, x2, and y2 are 4 coordinates of the position of the defect in the frame appearance area image, and score is the confidence of the defect.
The invention mainly utilizes the improved characteristic pyramid (FPN) network and the deep learning model of the backbone network to accurately identify the appearance difference of the electronic equipment frames of the second-hand electronic equipment such as a mobile phone.
In an embodiment of the method for detecting the frame appearance flaws of the electronic device, the second device is configured to extract a frame appearance area image of the electronic device from an appearance image of the electronic device in a manner of Unet instance segmentation.
Here, the frame appearance region image can be obtained quickly and efficiently by the Unet instance division.
In an embodiment of the method for detecting the appearance flaws of the frame of the electronic device, the front 2 layers of the backbone network adopt a res structure, and the rear 2 layers of the network adopt an acceptance structure.
In an embodiment of the method for detecting the appearance defect of the frame of the electronic device, the fourth device is further configured to identify whether a confidence of the defect detection result is greater than a first preset threshold, and if the confidence is greater than the first preset threshold, output result information including a defect type of the frame of the electronic device and a position of the defect in the frame of the electronic device.
Here, the defect types of the electronic device bezel may sequentially include a shallow scratch, a hard scratch, and a chipping type, which are sequentially increasing in grade.
In this embodiment, by identifying the confidence of the flaw detection result, a reliable result can be screened from the flaw detection result and output.
In an embodiment of the method for detecting the appearance defects of the frame of the electronic device, the method further includes a fifth apparatus including:
a fifth device, configured to preset a model of the FPN network combined with the backbone network and initial model parameters thereof;
a fifth second device, configured to input the frame appearance area image of the sample electronic device into a model combining a back bone network and an FPN network with current model parameters, to obtain a defect prediction result of the frame of the sample electronic device, where the defect prediction result includes: the defect type of the frame of the sample electronic equipment, the position of the defect in the frame of the sample electronic equipment and the confidence coefficient of the defect detection result;
a fifth third means for calculating a difference between the defect prediction result and a true defect result of the sample electronic device based on a preset objective function, and identifying whether the difference is greater than a second preset threshold, if the difference is greater than the second preset threshold, executing a fifth fourth means for restarting execution from the fifth second means after updating the model parameters of the FPN network in combination with the backbone network based on the difference;
and if the difference is smaller than or equal to a second preset threshold, executing a fifth device, and taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
And circularly training the model of the FPN network combined with the backbone network by identifying whether the difference value is greater than a second preset threshold, so as to obtain a reliable model.
In an embodiment of the method for detecting the frame appearance flaws of the electronic device, the second device is configured to scale the length direction of the frame appearance area image and fill the width direction of the frame appearance area image.
In the processing, the long side of the frame appearance area image is scaled, and the short side is filled, so as to obtain an image with the frame appearance area image adjusted to have the same length and width.
The present invention also provides a computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
step S1, acquiring an appearance image of the electronic equipment;
step S2, extracting a frame appearance area image of the electronic device from the appearance image of the electronic device, and adjusting the frame appearance area image to an image with the same length and width;
step S3, inputting the adjusted frame appearance area image into a model of the FPN network combined with the backbone network after training is finished;
step S4, receiving and outputting a defect detection result of the frame appearance area of the electronic device from the model of the FPN network combined with the backbone network, where the defect detection result includes: the defect detection method comprises the steps of detecting defects of a frame of the electronic device, determining the positions of the defects in the frame of the electronic device and determining the confidence of the defect detection result.
The present invention also provides a computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
step S1, acquiring an appearance image of the electronic equipment;
step S2, extracting a frame appearance area image of the electronic device from the appearance image of the electronic device, and adjusting the frame appearance area image to an image with the same length and width;
step S3, inputting the adjusted frame appearance area image into a model of the FPN network combined with the backbone network after training is finished;
step S4, receiving and outputting a defect detection result of the frame appearance area of the electronic device from the model of the FPN network combined with the backbone network, where the defect detection result includes: the defect detection method comprises the steps of detecting defects of a frame of the electronic device, determining the positions of the defects in the frame of the electronic device and determining the confidence of the defect detection result.
For details of embodiments of each device and storage medium of the present invention, reference may be made to corresponding parts of each method embodiment, and details are not described herein again.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
It should be noted that the present invention may be implemented in software and/or in a combination of software and hardware, for example, as an Application Specific Integrated Circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention may be executed by a processor to implement the steps or functions described above. Also, the software programs (including associated data structures) of the present invention can be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Further, some of the steps or functions of the present invention may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present invention can be applied as a computer program product, such as computer program instructions, which when executed by a computer, can invoke or provide the method and/or technical solution according to the present invention through the operation of the computer. Program instructions which invoke the methods of the present invention may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the invention herein comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or solution according to embodiments of the invention as described above.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (14)
1. A method for detecting appearance defects of a frame of an electronic device comprises the following steps:
acquiring an appearance image of the electronic equipment;
extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the frame appearance area image into an image with the same length and width;
inputting the adjusted frame appearance area image into a model of combining the FPN network and the backbone network after training is finished;
receiving a defect detection result of a frame appearance area of the electronic device, which is output from a model combining the FPN network and the backbone network, wherein the defect detection result comprises: the defect detection method comprises the steps of detecting defects of a frame of the electronic device, determining the positions of the defects in the frame of the electronic device and determining the confidence of the defect detection result.
2. The method of claim 1, wherein extracting a bezel appearance area image of the electronic device from the appearance image of the electronic device comprises:
and extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment by adopting a Unet instance segmentation mode.
3. The method of claim 1, wherein the first 2 layers of the backhaul network adopt res structure, and the last 2 layers of the network adopt initiation structure.
4. The method of claim 1, wherein after receiving the output defect detection result of the bezel appearance area of the electronic device from the model of the FPN network in combination with the backbone network, further comprising:
identifying whether a confidence level of the flaw detection result is greater than a first preset threshold,
and if the defect type is larger than the first preset threshold, outputting result information including the defect type of the frame of the electronic equipment and the position of the defect in the frame of the electronic equipment.
5. The method of claim 1, wherein before inputting the bezel appearance area image into the model of FPN network combined with backbone network, further comprising:
presetting a model of combining an FPN network with a backbone network and initial model parameters thereof;
inputting the frame appearance area image of the sample electronic equipment into a FPN network with current model parameters and combining with a backbone network model to obtain a defect prediction result of the frame of the sample electronic equipment, wherein the defect prediction result comprises the following steps: the defect type of the frame of the sample electronic equipment, the position of the defect in the frame of the sample electronic equipment and the confidence coefficient of the defect detection result;
calculating a difference between the flaw prediction result and a true flaw result of the sample electronic device based on a preset objective function, identifying whether the difference is greater than a second preset threshold,
if the difference value is larger than a second preset threshold value, a fourth step of executing from the second step again after updating the model parameters of the FPN network combined with the backbone network based on the difference value;
and if the difference is smaller than or equal to a second preset threshold, step five, taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
6. The method of claim 1, wherein adjusting the bezel appearance area image to an image of the same length and width comprises:
and zooming the length direction of the frame appearance area image, and filling the width direction of the frame appearance area image.
7. An electronic device bezel appearance flaw detection apparatus, wherein the apparatus comprises:
the device comprises a first device, a second device and a third device, wherein the first device is used for acquiring an appearance image of the electronic equipment;
the second device is used for extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment and adjusting the frame appearance area image into an image with the same length and width;
the third device is used for inputting the adjusted image of the frame appearance area into a model of the FPN network combined with the backbone network after training is finished;
a fourth device, configured to receive a defect detection result of a bezel appearance area of an electronic device from a model combining the FPN network and the backbone network, where the defect detection result includes: the defect detection method comprises the steps of detecting defects of a frame of the electronic device, determining the positions of the defects in the frame of the electronic device and determining the confidence of the defect detection result.
8. The apparatus of claim 7, wherein the second means is configured to extract a frame appearance region image of the electronic device from the appearance image of the electronic device by using a Unet instance segmentation method.
9. The apparatus of claim 7, wherein the front 2 layers of the backhaul network adopt res structure, and the back 2 layers of the network adopt initiation structure.
10. The apparatus of claim 7, wherein the fourth means is further configured to identify whether a confidence of the defect detection result is greater than a first preset threshold, and if so, output result information including a defect type of a bezel of the electronic device and a position of the defect in the bezel of the electronic device.
11. The apparatus of claim 7, further comprising a fifth apparatus comprising:
a fifth device, configured to preset a model of the FPN network combined with the backbone network and initial model parameters thereof;
a fifth second device, configured to input the frame appearance area image of the sample electronic device into a model combining a back bone network and an FPN network with current model parameters, to obtain a defect prediction result of the frame of the sample electronic device, where the defect prediction result includes: the defect type of the frame of the sample electronic equipment, the position of the defect in the frame of the sample electronic equipment and the confidence coefficient of the defect detection result;
a fifth third means for calculating a difference between the defect prediction result and a true defect result of the sample electronic device based on a preset objective function, and identifying whether the difference is greater than a second preset threshold, if the difference is greater than the second preset threshold, executing a fifth fourth means for restarting execution from the fifth second means after updating the model parameters of the FPN network in combination with the backbone network based on the difference;
and if the difference is smaller than or equal to a second preset threshold, executing a fifth device, and taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
12. The apparatus of claim 7, wherein the second means is configured to scale a length direction of the border appearance area image and fill a width direction of the border appearance area image.
13. A computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring an appearance image of the electronic equipment;
extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the frame appearance area image into an image with the same length and width;
inputting the adjusted frame appearance area image into a model of combining the FPN network and the backbone network after training is finished;
receiving a defect detection result of a frame appearance area of the electronic device, which is output from a model combining the FPN network and the backbone network, wherein the defect detection result comprises: the defect detection method comprises the steps of detecting defects of a frame of the electronic device, determining the positions of the defects in the frame of the electronic device and determining the confidence of the defect detection result.
14. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
acquiring an appearance image of the electronic equipment;
extracting a frame appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the frame appearance area image into an image with the same length and width;
inputting the adjusted frame appearance area image into a model of combining the FPN network and the backbone network after training is finished;
receiving a defect detection result of a frame appearance area of the electronic device, which is output from a model combining the FPN network and the backbone network, wherein the defect detection result comprises: the defect detection method comprises the steps of detecting defects of a frame of the electronic device, determining the positions of the defects in the frame of the electronic device and determining the confidence of the defect detection result.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911032912.6A CN110827246A (en) | 2019-10-28 | 2019-10-28 | Electronic equipment frame appearance flaw detection method and equipment |
| PCT/CN2020/120876 WO2021082920A1 (en) | 2019-10-28 | 2020-10-14 | Method and device for detecting border appearance defects of electronic device |
| JP2022502023A JP2022539909A (en) | 2019-10-28 | 2020-10-14 | Electronic device frame appearance defect inspection method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911032912.6A CN110827246A (en) | 2019-10-28 | 2019-10-28 | Electronic equipment frame appearance flaw detection method and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110827246A true CN110827246A (en) | 2020-02-21 |
Family
ID=69551291
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911032912.6A Pending CN110827246A (en) | 2019-10-28 | 2019-10-28 | Electronic equipment frame appearance flaw detection method and equipment |
Country Status (3)
| Country | Link |
|---|---|
| JP (1) | JP2022539909A (en) |
| CN (1) | CN110827246A (en) |
| WO (1) | WO2021082920A1 (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021082918A1 (en) * | 2019-10-28 | 2021-05-06 | 上海悦易网络信息技术有限公司 | Screen appearance defect detection method and device |
| WO2021082920A1 (en) * | 2019-10-28 | 2021-05-06 | 上海悦易网络信息技术有限公司 | Method and device for detecting border appearance defects of electronic device |
| US11798250B2 (en) | 2019-02-18 | 2023-10-24 | Ecoatm, Llc | Neural network based physical condition evaluation of electronic devices, and associated systems and methods |
| US11843206B2 (en) | 2019-02-12 | 2023-12-12 | Ecoatm, Llc | Connector carrier for electronic device kiosk |
| US11922467B2 (en) | 2020-08-17 | 2024-03-05 | ecoATM, Inc. | Evaluating an electronic device using optical character recognition |
| US11989710B2 (en) | 2018-12-19 | 2024-05-21 | Ecoatm, Llc | Systems and methods for vending and/or purchasing mobile phones and other electronic devices |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160322079A1 (en) * | 2014-02-05 | 2016-11-03 | Avatar Merger Sub II, LLC | Method for real time video processing involving changing a color of an object on a human face in a video |
| CN106875381A (en) * | 2017-01-17 | 2017-06-20 | 同济大学 | A kind of phone housing defect inspection method based on deep learning |
| CN109711474A (en) * | 2018-12-24 | 2019-05-03 | 中山大学 | A kind of aluminium material surface defects detection algorithm based on deep learning |
| CN109859190A (en) * | 2019-01-31 | 2019-06-07 | 北京工业大学 | A kind of target area detection method based on deep learning |
| CN109886077A (en) * | 2018-12-28 | 2019-06-14 | 北京旷视科技有限公司 | Image-recognizing method, device, computer equipment and storage medium |
| CN110378420A (en) * | 2019-07-19 | 2019-10-25 | Oppo广东移动通信有限公司 | A kind of image detecting method, device and computer readable storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11210777B2 (en) * | 2016-04-28 | 2021-12-28 | Blancco Technology Group IP Oy | System and method for detection of mobile device fault conditions |
| CN108918528B (en) * | 2018-06-01 | 2023-08-01 | 深圳回收宝科技有限公司 | Terminal detection method, device and storage medium |
| CN109859163A (en) * | 2018-12-19 | 2019-06-07 | 重庆邮电大学 | A kind of LCD defect inspection method based on feature pyramid convolutional neural networks |
| CN109800709A (en) * | 2019-01-18 | 2019-05-24 | 张琪培 | A kind of automatic rotary cabinet automatic recognition system and method based on deep learning |
| CN110827246A (en) * | 2019-10-28 | 2020-02-21 | 上海悦易网络信息技术有限公司 | Electronic equipment frame appearance flaw detection method and equipment |
-
2019
- 2019-10-28 CN CN201911032912.6A patent/CN110827246A/en active Pending
-
2020
- 2020-10-14 WO PCT/CN2020/120876 patent/WO2021082920A1/en active Application Filing
- 2020-10-14 JP JP2022502023A patent/JP2022539909A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160322079A1 (en) * | 2014-02-05 | 2016-11-03 | Avatar Merger Sub II, LLC | Method for real time video processing involving changing a color of an object on a human face in a video |
| CN106875381A (en) * | 2017-01-17 | 2017-06-20 | 同济大学 | A kind of phone housing defect inspection method based on deep learning |
| CN109711474A (en) * | 2018-12-24 | 2019-05-03 | 中山大学 | A kind of aluminium material surface defects detection algorithm based on deep learning |
| CN109886077A (en) * | 2018-12-28 | 2019-06-14 | 北京旷视科技有限公司 | Image-recognizing method, device, computer equipment and storage medium |
| CN109859190A (en) * | 2019-01-31 | 2019-06-07 | 北京工业大学 | A kind of target area detection method based on deep learning |
| CN110378420A (en) * | 2019-07-19 | 2019-10-25 | Oppo广东移动通信有限公司 | A kind of image detecting method, device and computer readable storage medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11989710B2 (en) | 2018-12-19 | 2024-05-21 | Ecoatm, Llc | Systems and methods for vending and/or purchasing mobile phones and other electronic devices |
| US11843206B2 (en) | 2019-02-12 | 2023-12-12 | Ecoatm, Llc | Connector carrier for electronic device kiosk |
| US11798250B2 (en) | 2019-02-18 | 2023-10-24 | Ecoatm, Llc | Neural network based physical condition evaluation of electronic devices, and associated systems and methods |
| WO2021082918A1 (en) * | 2019-10-28 | 2021-05-06 | 上海悦易网络信息技术有限公司 | Screen appearance defect detection method and device |
| WO2021082920A1 (en) * | 2019-10-28 | 2021-05-06 | 上海悦易网络信息技术有限公司 | Method and device for detecting border appearance defects of electronic device |
| US11922467B2 (en) | 2020-08-17 | 2024-03-05 | ecoATM, Inc. | Evaluating an electronic device using optical character recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2022539909A (en) | 2022-09-13 |
| WO2021082920A1 (en) | 2021-05-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110675399A (en) | Screen appearance flaw detection method and equipment | |
| CN110827249A (en) | Electronic equipment backboard appearance flaw detection method and equipment | |
| CN110827246A (en) | Electronic equipment frame appearance flaw detection method and equipment | |
| US9697416B2 (en) | Object detection using cascaded convolutional neural networks | |
| CN110796646A (en) | Method and device for detecting defects of screen area of electronic device | |
| US11682225B2 (en) | Image processing to detect a rectangular object | |
| CN111175318A (en) | Screen scratch fragmentation detection method and equipment | |
| TW201447775A (en) | Method and system for recognizing information | |
| CN110796647A (en) | Method and device for detecting defects of screen area of electronic device | |
| US8374454B2 (en) | Detection of objects using range information | |
| CN110827244A (en) | Method and equipment for detecting appearance flaws of electronic equipment | |
| CN111291661B (en) | Method and equipment for identifying text content of icon in screen | |
| US8254690B2 (en) | Information processing apparatus, information processing method, and program | |
| CN110796669A (en) | Vertical frame positioning method and equipment | |
| CN110348392B (en) | Vehicle matching method and device | |
| CN110708568B (en) | Video content mutation detection method and device | |
| CN111210473A (en) | Mobile phone contour positioning method and equipment | |
| CN111401238A (en) | Method and device for detecting character close-up segments in video | |
| US11728914B2 (en) | Detection device, detection method, and program | |
| CN110728193B (en) | Method and device for detecting richness characteristics of face image | |
| CN106485246B (en) | Character identifying method and device | |
| CN112052863B (en) | Image detection method and device, computer storage medium and electronic equipment | |
| CN112801987B (en) | Mobile phone part abnormity detection method and equipment | |
| CN115004245A (en) | Target detection method, target detection device, electronic equipment and computer storage medium | |
| CN111985423A (en) | Living body detection method, living body detection device, living body detection equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 1101-1103, No. 433, Songhu Road, Yangpu District, Shanghai Applicant after: Shanghai wanwansheng Environmental Protection Technology Group Co.,Ltd. Address before: Room 1101-1103, No. 433, Songhu Road, Yangpu District, Shanghai Applicant before: SHANGHAI YUEYI NETWORK INFORMATION TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200221 |
|
| RJ01 | Rejection of invention patent application after publication |