WO2021147386A1

WO2021147386A1 - Screen scratch and crack detection method and device

Info

Publication number: WO2021147386A1
Application number: PCT/CN2020/120889
Authority: WO
Inventors: 陈敏
Original assignee: 上海万物新生环保科技集团有限公司
Priority date: 2020-01-21
Filing date: 2020-10-14
Publication date: 2021-07-29
Also published as: CN111175318A

Abstract

A screen scratch and crack detection method and device. On the basis of image features corresponding to a yellow screen image and a black screen image, respectively, target candidate frames, of which the target categories are a scratch category and a crack category, in the yellow screen image and the black screen image are obtained, so that scratches or cracks on screens of devices such as mobile phones can be accurately recognized, increasing the evaluation and recycling efficiency of smart devices such as mobile phones.

Description

Screen scratch and chipping detection method and equipment

Technical field

The invention relates to the field of computers, in particular to a method and equipment for detecting screen scratches and chipping.

Background technique

Existing methods for detecting screen scratches on mobile phones and other devices are manual methods, which are time-consuming and labor-intensive, and affect the efficiency of mobile phones and other smart devices such as valuation and recycling.

Summary of the invention

An object of the present invention is to provide a method and equipment for detecting screen scratches and chipping.

According to one aspect of the present invention, there is provided a method for detecting screen scratches and chipping, the method including:

Control the screen to display a full-screen yellow image below the preset exposure value, and take a yellow screen image based on the outline position of the screen;

Control the screen to display a full-screen black image higher than the preset exposure value, and take a black screen image based on the outline position of the screen;

Inputting the yellow screen image into a convolutional neural network, and extracting image features corresponding to the yellow screen image; inputting the black screen image into a convolutional neural network, and extracting image features corresponding to the black screen image;

Based on the image features corresponding to the yellow screen image and the black screen image, respectively, target candidate frames in the yellow screen image and the black screen image are obtained in which the target categories are the scratch pattern category and the broken crack category.

Further, in the above method, the convolutional neural network is a resnext101 convolutional neural network.

Further, in the above method, based on the image features corresponding to the yellow screen image and the black screen image, respectively, the yellow screen image and the black screen image are obtained, and the target categories are the target candidate frames of the scratch pattern category and the broken crack category. ,include:

Based on the image features corresponding to the yellow screen image, and through the FPN method, the corresponding multi-layer feature layers of different scales corresponding to the yellow screen image are obtained; based on the image features corresponding to the black screen image and the corresponding black screen image Image features, and using the FPN method to obtain multi-layer feature layers of different scales corresponding to the corresponding black screen image;

The target candidate frame in the yellow screen image is extracted through the RPN (Region Proposal Network) network on the multi-layer feature layers of different scales corresponding to the yellow screen image, and each target in the yellow screen image is preset The candidate frame has the probability value of scratches and cracks; the target candidate frame in the black screen image is extracted by the RPN network on the multi-layer feature layers of different scales corresponding to the black screen image, and the preset The probability value of scratches and cracks in each target candidate frame in the black screen image;

Selecting the first preset number of target candidate frames in the yellow screen image with a larger probability value; selecting the first preset number of target candidate frames in the black screen image with a larger probability value;

The first preset number of target candidate frames in the yellow screen image are input into the classification neural network, and each target candidate frame in the first preset number of target candidate frames in the yellow screen image corresponding to the output is obtained The probability values belonging to the background category, the scratch pattern category and the broken crack category respectively; input the previously preset number of target candidate frames in the black screen image into the classification neural network, and obtain the corresponding output black screen image The probability value of each target candidate frame in the previously preset number of target candidate frames belonging to the background category, the scratch pattern category and the broken crack category respectively;

Determine the corresponding category with a larger probability value of each target candidate frame as the initial category of the target candidate frame;

If it is determined that the probability value of the initial category of the target candidate frame of the initial category is greater than the preset probability threshold, then the initial category is determined as the target category of the target candidate frame;

The output determines the target candidate frame with the target category as scratch pattern category and broken crack category.

Further, in the above method, outputting the target candidate frames whose target categories are determined to be the scratch pattern category and the broken crack category include:

The target candidate frames in the yellow screen image with the positions of the determined target categories overlapped are sorted in descending order based on the probability value to obtain the first sorting queue, and the target candidate frame with the highest probability value in the first sorting queue is used as the first reference Candidate frame, if the overlapping area of each target candidate frame in the subsequent queues in the first sorting queue and the first reference candidate frame exceeds the threshold of the area of the first reference candidate frame of the preset ratio, then The target candidate frame and its corresponding target category are deleted; the target candidate frames in the black screen image where the target category is determined overlapped are sorted in descending order based on the probability value to obtain a second sorting queue, and the second sorting queue is placed in the second sorting queue. The target candidate frame with the highest probability value is used as the second reference candidate frame. If the overlapping area of each subsequent target candidate frame in the second sorting queue and the second reference candidate frame exceeds the preset ratio of the second reference candidate frame The threshold of the area of, delete the target candidate frame and its corresponding target category;

Further, in the above method, the classification neural network is a fully connected layer classification neural network.

According to another aspect of the present invention, there is also provided a screen scratch and chipping detection device, wherein the device includes:

The display and shooting device is used to control the screen to display a full-screen yellow image below the preset exposure value, and shoot the yellow screen image based on the outline position of the screen; control the screen to display a full-screen black image higher than the preset exposure value, based on The outline position of the screen, taking a black screen image;

A feature extraction device for inputting the yellow screen image into a convolutional neural network, and extracting image features corresponding to the yellow screen image; inputting the black screen image into the convolutional neural network, and extracting the corresponding black screen image Image characteristics;

The recognition device is configured to obtain target candidate frames in the yellow screen image and the black screen image whose target categories are the scratch pattern category and the broken crack category based on the image features corresponding to the yellow screen image and the black screen image respectively.

Further, in the above device, the convolutional neural network is a resnext101 convolutional neural network.

Further, in the above-mentioned equipment, the identification device is used for:

The target candidate frame in the yellow screen image is extracted through the RPN network on the multi-layer feature layers of different scales corresponding to the yellow screen image, and each target candidate frame in the yellow screen image is preset to have scratches The probability value of grains and cracks; the target candidate frame in the black screen image is extracted on the multi-layer feature layers of different scales corresponding to the black screen image through the RPN network, and the target candidate frame in the black screen image is preset The probability value of scratches and cracks in each target candidate frame;

Further, in the above device, the classification neural network is a fully connected layer classification neural network.

According to another aspect of the present invention, there is also provided a computing-based device, which includes:

Processor; and

A memory arranged to store computer-executable instructions which, when executed, cause the processor to:

According to another aspect of the present invention, there is also provided a computer-readable storage medium having computer-executable instructions stored thereon, wherein, when the computer-executable instructions are executed by a processor, the processor:

Compared with the prior art, the present invention obtains that the target categories in the yellow screen image and the black screen image are scratch marks and cracks based on the corresponding image features of the yellow screen image and the black screen image. The target candidate frame can accurately identify scratches or cracks on the screen of mobile phones and other devices, and can improve the efficiency of mobile phone and other smart devices such as valuation and recycling.

Description of the drawings

By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more apparent:

FIG. 1 shows a flowchart of a method for detecting screen scratches and chipping according to an embodiment of the present invention.

The same or similar reference signs in the drawings represent the same or similar components.

Detailed ways

The present invention will be described in further detail below in conjunction with the accompanying drawings.

In a typical configuration of this application, the terminal, the equipment of the service network, and the trusted party all include one or more processors (CPU), input/output interfaces, network interfaces, and memory.

The memory may include non-permanent memory in computer readable media, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer readable media.

Computer-readable media includes permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. The information can be computer readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. According to the definition in this article, computer-readable media does not include non-transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

As shown in FIG. 1, the present invention provides a method for detecting screen scratches and chipping. The method includes:

Step S1, controlling the screen to display a full-screen yellow image lower than the preset exposure value, and taking a yellow screen image based on the outline position of the screen;

Here, for devices with white outlines, such as mobile phones, PADs, etc., by acquiring yellow screen images, the accuracy of the identification of scratches and cracks on the screen of such devices can be guaranteed;

Step S2, controlling the screen to display a full-screen black image higher than the preset exposure value, and taking a black screen image based on the outline position of the screen;

Here, the purpose of taking pictures with high and low exposure values is: high-exposure pictures are conducive to shooting dark screen surface textures, but for bright screen surface textures it is easy to cause overexposure problems, so low-exposure pictures need to be used for auxiliary detection;

The purpose of shooting black and yellow pictures is: through experiments, we know that different types of textures are different in the different background color pictures, so we choose black and yellow pictures with better experimental results as the background;

Step S3, input the yellow screen image into the convolutional neural network, and extract the image features corresponding to the yellow screen image; input the black screen image into the convolutional neural network, and extract the image features corresponding to the black screen image ；

Here, the convolutional neural network may be a resnext101 convolutional neural network to extract accurate image features;

Step S4, based on the image features corresponding to the yellow screen image and the black screen image, respectively, obtain target candidate frames in the yellow screen image and the black screen image whose target categories are the scratch pattern category and the broken crack category.

Here, the present invention obtains target candidate frames in the yellow screen image and the black screen image whose target categories are the scratch pattern category and the broken crack category based on the image features corresponding to the yellow screen image and the black screen image, respectively. It can accurately identify scratches or cracks on the screen of mobile phones and other devices, and can improve the efficiency of valuation and recycling of smart devices such as mobile phones.

In an embodiment of the screen scratch detection method of the present invention, in step S4, based on the image characteristics corresponding to the yellow screen image and the black screen image, respectively, the yellow screen image and the black screen image are obtained, and the target category is scratch The target candidate frames of the mark type and the crack type include:

Step S41: Based on the image features corresponding to the yellow screen image, and through the FPN (feature pyramid networks) method, obtain the corresponding multi-layer feature layers of different scales corresponding to the yellow screen image; Image features and image features corresponding to the black screen image, and using the FPN method to obtain the corresponding multi-layer feature layers of different scales corresponding to the black screen image;

Step S42: Extract target candidate frames in the yellow screen image on the multi-layer feature layers of different scales corresponding to the yellow screen image through the RPN network, and preset each target candidate frame in the yellow screen image There is a probability value of scratches and cracks; the target candidate frame in the black screen image is extracted by the PRN network on the multi-layer feature layers of different scales corresponding to the black screen image, and the black screen is preset The probability value of scratches and cracks in each target candidate frame in the image;

Step S43, selecting the first preset number of target candidate frames in the yellow screen image with a larger probability value; selecting the first preset number of target candidate frames in the black screen image with a larger probability value;

Here, the first 1000 target candidate frames in the yellow screen image with a larger probability value may be selected; the first 1000 target candidate frames in the black screen image with a larger probability value may be selected;

Step S44: Input the previously preset number of target candidate frames in the yellow screen image into the classification neural network, and obtain each corresponding output of the first preset number of target candidate frames in the yellow screen image The target candidate frames respectively belong to the probability values of the background category, the scratch pattern category and the broken crack category; input the previously preset number of target candidate frames in the black screen image into the classification neural network, and obtain the corresponding output The probability value that each target candidate frame in the previously preset number of target candidate frames in the black screen image belongs to the background category, the scratch pattern category and the broken crack category respectively;

Here, the classification neural network may be a fully connected layer classification neural network to obtain reliable classification;

Step S45: Determine the corresponding category with a larger probability value of each target candidate frame as the initial category of the target candidate frame;

Here, for example, if the neural network outputs a target candidate frame a with a background category probability value of 0.2, a scratch pattern category has a probability value of 0.3, and a broken crack category has a probability value of 0.5, then the target candidate frame a The initial category is the crack category;

For another example, the neural network outputs the probability value of the background category of a certain target candidate frame b as 0.1, the probability value of the scratch pattern category is 0.2, and the probability value of the broken crack category is 0.7, then the initial value of the target candidate frame b The category is broken and cracked;

Step S46, if it is determined that the probability value of the initial category of the target candidate frame of the initial category is greater than the preset probability threshold, then the initial category is determined as the target category of the target candidate frame;

Here, for example, the preset probability threshold is 0.6,

The neural network outputs that the initial category of a certain target candidate frame a is the broken crack category, and the probability value of the broken crack category is 0.5. Since the predetermined probability threshold of 0.6 is not exceeded, the initial category of the broken crack category of the target candidate frame a The category cannot be used as the target category;

For another example, the neural network outputs the initial category of a certain target candidate frame b as the broken crack category, and the probability value of the broken crack category is 0.7. Since the predetermined probability threshold of 0.6 is exceeded, the broken crack category of the target candidate frame b The initial category of can be used as the target category;

Step S47, outputting a target candidate frame whose target categories are determined to be the scratch pattern category and the broken crack category.

Here, in this embodiment, the initial category of the target candidate frame is determined, and then the target candidate frame of the determined target category is filtered from the target candidate frame of the determined initial category, which can further reliably and accurately identify the screen of the mobile phone and other devices. Scratches or cracks.

In an embodiment of the screen scratch and chipping detection method of the present invention, in step S47, outputting target candidate frames whose target categories are determined to be the scratch pattern category and the broken crack category, including:

Step S471: Arrange the target candidate frames in the yellow screen image that overlap the target categories in descending order based on the probability value to obtain a first sorting queue, and use the target candidate frame with the highest probability value in the first sorting queue as The first reference candidate frame, if the overlapping area of each target candidate frame in the subsequent queues in the first sorting queue and the first reference candidate frame exceeds the threshold value of the area of the first reference candidate frame of the preset ratio , The target candidate frame and its corresponding target category are deleted; the target candidate frames in the black screen image where the target category is determined overlapped are arranged in descending order based on the probability value to obtain the second sorting queue, and the second sorting queue is obtained. The target candidate frame with the highest probability value in the sorting queue is used as the second reference candidate frame. If the overlapping area of each subsequent target candidate frame in the second sorting queue and the second reference candidate frame exceeds the preset ratio of the second reference candidate frame The threshold of the area of the reference candidate frame, the target candidate frame and its corresponding target category are deleted;

Step S472, outputting the target candidate frames whose target categories are determined to be the scratch pattern category and the broken crack category.

Here, the preset ratio threshold may be 0.7. When the overlapping area of each target candidate frame and the reference candidate frame in the subsequent queues in the sorting queue exceeds 0.7 of the area of the reference candidate frame, Then delete the target candidate frame and its corresponding target category;

In this embodiment, by further filtering and deleting each subsequent target candidate frame whose overlapping area exceeds the threshold value of the area of the reference candidate frame of the preset ratio and the area of the quasi-candidate frame, it is possible to ensure that the output reliably determines that the target category is scratched. The target candidate frame of the category and the crack category.

The present invention provides a screen scratch and chipping detection device, which includes:

The display shooting device is used to control the screen to display a full-screen yellow image below the preset exposure value, and shoot the yellow screen image based on the outline position of the screen; control the screen to display a full-screen black image higher than the preset exposure value, based on The outline position of the screen, taking a black screen image;

The recognition device is configured to obtain target candidate frames in the yellow screen image and the black screen image whose target categories are the scratch pattern category and the broken crack category based on the image features corresponding to the yellow screen image and the black screen image, respectively.

In an embodiment of the screen scratch and chipping detection method of the present invention, the identification device is used for:

Based on the image features corresponding to the yellow screen image, and through the FPN (feature pyramid networks) method, the corresponding yellow screen image corresponding to the multi-layer feature layer of different scales is obtained; based on the image features corresponding to the black screen image and Image features corresponding to the black screen image, and using the FPN method to obtain multi-layer feature layers of different scales corresponding to the black screen image;

Here, the classification neural network may be a fully connected layer classification neural network;

Here, for example, the preset probability threshold is 0.6,

For another example, the neural network outputs that the initial category of a certain target candidate frame b is the broken crack category, and the probability value of the broken crack category is 0.7. Since the preset probability threshold of 0.6 is not exceeded, the broken crack of the target candidate frame b The initial category of the category can be used as the target category;

The target candidate frames in the yellow screen image with the positions of the determined target categories overlapped are sorted in descending order based on the probability value to obtain the first sorting queue, and the target candidate frame with the highest probability value in the first sorting queue is used as the first reference Candidate frame, if the overlapping area of each target candidate frame in the subsequent queues in the first sorting queue and the first reference candidate frame exceeds the threshold of the area of the first reference candidate frame of the preset ratio, then The target candidate frame and its corresponding target category are deleted; the target candidate frames in the black screen image whose positions where the target category is determined overlap are arranged in descending order based on the probability value to obtain a second sorting queue, and the second sorting queue is placed in the second sorting queue. The target candidate frame with the highest probability value is used as the second reference candidate frame. If the overlapping area of each subsequent target candidate frame in the second sorting queue and the second reference candidate frame exceeds the preset ratio of the second reference candidate frame The threshold of the area of, delete the target candidate frame and its corresponding target category;

Processor; and

Control the screen to display a full-screen black image higher than the preset exposure value, and shoot a black screen image based on the outline position of the screen;

For details of the device and storage medium embodiments of the present invention, please refer to the corresponding parts of the method embodiments for details, which will not be repeated here.

Obviously, those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application also intends to include these modifications and variations.

It should be noted that the present invention can be implemented in software and/or a combination of software and hardware. For example, it can be implemented by an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In an embodiment, the software program of the present invention may be executed by a processor to realize the above-mentioned steps or functions. Similarly, the software program (including related data structure) of the present invention can be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present invention may be implemented by hardware, for example, as a circuit that cooperates with a processor to execute each step or function.

In addition, a part of the present invention can be applied as a computer program product, such as a computer program instruction, when it is executed by a computer, through the operation of the computer, the method and/or technical solution according to the present invention can be invoked or provided. The program instructions for invoking the method of the present invention may be stored in a fixed or removable recording medium, and/or transmitted through a data stream in a broadcast or other signal-bearing medium, and/or stored in accordance with the Said program instructions run in the working memory of the computer equipment. Here, an embodiment according to the present invention includes a device including a memory for storing computer program instructions and a processor for executing the program instructions, wherein, when the computer program instructions are executed by the processor, trigger The operation of the device is based on the aforementioned methods and/or technical solutions according to multiple embodiments of the present invention.

For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description, and therefore it is intended to fall within the claims. All changes within the meaning and scope of the equivalent elements of are included in the present invention. Any reference signs in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claims can also be implemented by one unit or device through software or hardware. The first and second words are used to denote names, but do not denote any specific order.

Claims

A method for detecting broken screen scratches, wherein the method includes:

Control the screen to display a full-screen yellow image below the preset exposure value, and take a yellow screen image based on the outline position of the screen;

Control the screen to display a full-screen black image higher than the preset exposure value, and take a black screen image based on the outline position of the screen;

Inputting the yellow screen image into a convolutional neural network, and extracting image features corresponding to the yellow screen image; inputting the black screen image into a convolutional neural network, and extracting image features corresponding to the black screen image;

Based on the image features corresponding to the yellow screen image and the black screen image, respectively, target candidate frames in the yellow screen image and the black screen image are obtained in which the target categories are the scratch pattern category and the broken crack category.
The method according to claim 1, wherein the convolutional neural network is a resnext101 convolutional neural network.
The method according to claim 1, wherein the yellow screen image and the black screen image are obtained based on the image characteristics corresponding to the yellow screen image and the black screen image respectively, and the target categories are the scratch pattern category and the broken crack category The target candidate frame includes:

Based on the image features corresponding to the yellow screen image, and through the FPN method, the corresponding multi-layer feature layers of different scales corresponding to the yellow screen image are obtained; based on the image features corresponding to the black screen image and the corresponding black screen image Image features, and using the FPN method to obtain multi-layer feature layers of different scales corresponding to the corresponding black screen image;

The target candidate frame in the yellow screen image is extracted through the RPN network on the multi-layer feature layers of different scales corresponding to the yellow screen image, and each target candidate frame in the yellow screen image is preset to have scratches The probability value of grains and cracks; the target candidate frame in the black screen image is extracted on the multi-layer feature layers of different scales corresponding to the black screen image through the RPN network, and the target candidate frame in the black screen image is preset The probability value of scratches and cracks in each target candidate frame;

Selecting the first preset number of target candidate frames in the yellow screen image with a larger probability value; selecting the first preset number of target candidate frames in the black screen image with a larger probability value;

The first preset number of target candidate frames in the yellow screen image are input into the classification neural network, and each target candidate frame in the first preset number of target candidate frames in the yellow screen image corresponding to the output is obtained The probability values belonging to the background category, the scratch pattern category and the broken crack category respectively; input the previously preset number of target candidate frames in the black screen image into the classification neural network, and obtain the corresponding output black screen image The probability value of each target candidate frame in the previously preset number of target candidate frames belonging to the background category, the scratch pattern category and the broken crack category respectively;

Determine the corresponding category with a larger probability value of each target candidate frame as the initial category of the target candidate frame;

If it is determined that the probability value of the initial category of the target candidate frame of the initial category is greater than the preset probability threshold, then the initial category is determined as the target category of the target candidate frame;

The output determines the target candidate frame with the target category as scratch pattern category and broken crack category.
3. The method according to claim 3, wherein outputting the target candidate frames whose target categories are determined to be the scratch pattern category and the broken crack category comprises:

The target candidate frames in the yellow screen image with the positions of the determined target categories overlapped are sorted in descending order based on the probability value to obtain the first sorting queue, and the target candidate frame with the highest probability value in the first sorting queue is used as the first reference Candidate frame, if the overlapping area of each target candidate frame in the subsequent queues in the first sorting queue and the first reference candidate frame exceeds the threshold of the area of the first reference candidate frame of the preset ratio, then The target candidate frame and its corresponding target category are deleted; the target candidate frames in the black screen image whose positions where the target category is determined overlap are arranged in descending order based on the probability value to obtain a second sorting queue, and the second sorting queue is placed in the second sorting queue. The target candidate frame with the highest probability value is used as the second reference candidate frame. If the overlapping area of each subsequent target candidate frame in the second sorting queue and the second reference candidate frame exceeds the preset ratio of the second reference candidate frame The threshold of the area of, delete the target candidate frame and its corresponding target category;

The output determines the target candidate frame with the target category as scratch pattern category and broken crack category.
The method according to claim 1, wherein the classification neural network is a fully connected layer classification neural network.
A screen scratch and chipping detection device, wherein the device includes:

The display and shooting device is used to control the screen to display a full-screen yellow image below the preset exposure value, and shoot the yellow screen image based on the outline position of the screen; control the screen to display a full-screen black image higher than the preset exposure value, based on The outline position of the screen, taking a black screen image;

A feature extraction device for inputting the yellow screen image into a convolutional neural network, and extracting image features corresponding to the yellow screen image; inputting the black screen image into the convolutional neural network, and extracting the corresponding black screen image Image characteristics;

An identification device is used to obtain target candidate frames in the yellow screen image and the black screen image whose target categories are the scratch pattern category and the broken crack category based on the image features corresponding to the yellow screen image and the black screen image respectively.
The device according to claim 6, wherein the convolutional neural network is a resnext101 convolutional neural network.
The device according to claim 6, wherein the identification device is used for:

Based on the image features corresponding to the yellow screen image, and through the FPN method, the corresponding multi-layer feature layers of different scales corresponding to the yellow screen image are obtained; based on the image features corresponding to the black screen image and the corresponding black screen image Image features, and using the FPN method to obtain multi-layer feature layers of different scales corresponding to the corresponding black screen image;

The target candidate frame in the yellow screen image is extracted through the RPN network on the multi-layer feature layers of different scales corresponding to the yellow screen image, and each target candidate frame in the yellow screen image is preset to have scratches The probability value of grains and cracks; the target candidate frame in the black screen image is extracted on the multi-layer feature layers of different scales corresponding to the black screen image through the RPN network, and the target candidate frame in the black screen image is preset The probability value of scratches and cracks in each target candidate frame;

Selecting the first preset number of target candidate frames in the yellow screen image with a larger probability value; selecting the first preset number of target candidate frames in the black screen image with a larger probability value;

The first preset number of target candidate frames in the yellow screen image are input into the classification neural network, and each target candidate frame in the first preset number of target candidate frames in the yellow screen image corresponding to the output is obtained The probability values belonging to the background category, the scratch pattern category and the broken crack category respectively; input the previously preset number of target candidate frames in the black screen image into the classification neural network, and obtain the corresponding output black screen image The probability value of each target candidate frame in the previously preset number of target candidate frames belonging to the background category, the scratch pattern category and the broken crack category respectively;

Determine the corresponding category with a larger probability value of each target candidate frame as the initial category of the target candidate frame;

If it is determined that the probability value of the initial category of the target candidate frame of the initial category is greater than the preset probability threshold, then the initial category is determined as the target category of the target candidate frame;

The output determines the target candidate frame with the target category as scratch pattern category and broken crack category.
The device according to claim 8, wherein the identification device is used for:

The target candidate frames in the yellow screen image with the positions of the determined target categories overlapped are sorted in descending order based on the probability value to obtain the first sorting queue, and the target candidate frame with the highest probability value in the first sorting queue is used as the first reference Candidate frame, if the overlapping area of each target candidate frame in the subsequent queues in the first sorting queue and the first reference candidate frame exceeds the threshold of the area of the first reference candidate frame of the preset ratio, then The target candidate frame and its corresponding target category are deleted; the target candidate frames in the black screen image whose positions where the target category is determined overlap are arranged in descending order based on the probability value to obtain a second sorting queue, and the second sorting queue is placed in the second sorting queue. The target candidate frame with the highest probability value is used as the second reference candidate frame. If the overlapping area of each subsequent target candidate frame in the second sorting queue and the second reference candidate frame exceeds the preset ratio of the second reference candidate frame The threshold of the area of, delete the target candidate frame and its corresponding target category;

The output determines the target candidate frame with the target category as scratch pattern category and broken crack category.
The device according to claim 8, wherein the classification neural network is a fully connected layer classification neural network.
A computing-based device, which includes:

Processor; and

A memory arranged to store computer-executable instructions which, when executed, cause the processor to:

Control the screen to display a full-screen yellow image below the preset exposure value, and take a yellow screen image based on the outline position of the screen;

Control the screen to display a full-screen black image higher than the preset exposure value, and take a black screen image based on the outline position of the screen;

Inputting the yellow screen image into a convolutional neural network, and extracting image features corresponding to the yellow screen image; inputting the black screen image into a convolutional neural network, and extracting image features corresponding to the black screen image;

Based on the image features corresponding to the yellow screen image and the black screen image, respectively, target candidate frames in the yellow screen image and the black screen image are obtained in which the target categories are the scratch pattern category and the broken crack category.
A computer-readable storage medium having computer-executable instructions stored thereon, wherein, when the computer-executable instructions are executed by a processor, the processor:

Control the screen to display a full-screen yellow image below the preset exposure value, and take a yellow screen image based on the outline position of the screen;

Control the screen to display a full-screen black image higher than the preset exposure value, and take a black screen image based on the outline position of the screen;

Inputting the yellow screen image into a convolutional neural network, and extracting image features corresponding to the yellow screen image; inputting the black screen image into a convolutional neural network, and extracting image features corresponding to the black screen image;

Based on the image features corresponding to the yellow screen image and the black screen image, respectively, target candidate frames in the yellow screen image and the black screen image are obtained in which the target categories are the scratch pattern category and the broken crack category.