CN111340774A

CN111340774A - Image detection method, image detection device, computer equipment and storage medium

Info

Publication number: CN111340774A
Application number: CN202010113011.6A
Authority: CN
Inventors: 朱艳春; 常佳
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-02-24
Filing date: 2020-02-24
Publication date: 2020-06-26
Anticipated expiration: 2040-02-24
Also published as: CN111340774B

Abstract

The application provides an image detection method, an image detection device, computer equipment and a storage medium, and belongs to the field of image processing. According to the technical scheme provided by the embodiment of the application, the first picture shot when the target object is not in the abnormal condition is taken as a reference, the second image and the first image acquired in the target processing process are compared in real time, and whether the second image is the picture in the abnormal condition or not is determined based on the comparison result. Because the second image is obtained in real time, relevant personnel can obtain the time when the target object is abnormal in the detection of the image detection model at the first time, and make corresponding processing scheme adjustment, thereby improving the possibility and the safety of the target processing process and improving the processing effect.

Description

Image detection method, image detection device, computer equipment and storage medium

Technical Field

The present application relates to the field of image processing, and in particular, to an image detection method, an image detection apparatus, a computer device, and a storage medium.

Background

With the development of computer technology, the application range of image processing technology is more and more extensive, some abnormal situations often occur in the process of performing target processing on some target objects, and timely intervention is needed when the abnormal situations occur, so that excessive loss caused by the abnormal situations is avoided, wherein the target objects can generally refer to various entities, such as solid metal blocks, wood, and even human bodies; the target treatment may refer to deformation of a solid metal block, cutting of wood, and treatment of a human body; the abnormal condition may refer to cracks occurring after the deformation process of the solid metal block, cracks occurring after the cutting process of the wood, and some unexpected conditions occurring after the treatment process of the human body.

In the related art, image capturing is often performed before target processing is performed on a target object to obtain an initial state image of the target object, image capturing is performed after the target processing is performed to obtain a final state image of the target object, and whether an abnormality occurs in the target object is determined based on a difference between the initial state image and the final state image. However, in this processing method, it is only known whether the target object is abnormal or not, and the time when the abnormal condition occurs cannot be known, so that the controllability of the target processing process is poor, and an unsafe condition may occur in the process of implementing the target processing, and the processing effect is poor.

Disclosure of Invention

The embodiment of the application provides an image detection method, an image detection device, computer equipment and a storage medium, which can improve the processing effect on a target object. The technical scheme is as follows:

in one aspect, an image detection method is provided, and the method includes:

acquiring a first image of a target object, wherein the first image is used for representing the original state of the target object before target processing is carried out on the target object;

acquiring a second image of the target object in real time in the process of carrying out the target processing on the target object;

calling an image detection model, wherein the image detection model is obtained by training based on a positive sample image and a negative sample image obtained in a target processing process of a sample object, the positive sample image is used for indicating that the sample object is in an abnormal condition, and the negative sample image is used for indicating that the sample object is not in the abnormal condition;

inputting the first image and the second image into the image detection model, respectively extracting features of the first image and the second image by the image detection model, acquiring difference information between the images according to the extracted first image feature of the first image and the second image feature of the second image, predicting based on the difference information to obtain risk information of the second image, and determining the second image as a target image if the risk information meets a target condition, wherein the target image is used for indicating that the target object has the abnormal condition;

and determining the shooting time of the second image as the occurrence time of the abnormal condition.

In one aspect, an image detection apparatus is provided, the apparatus including:

the device comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a first image of a target object, and the first image is used for representing the original state of the target object before target processing is carried out on the target object;

the acquisition model is further used for acquiring a second image of the target object in real time in the process of carrying out the target processing on the target object;

the model calling module is used for calling an image detection model, the image detection model is obtained by training based on a positive sample image and a negative sample image which are obtained in the target processing process of a sample object, the positive sample image is used for indicating that the sample object is in an abnormal condition, and the negative sample image is used for indicating that the sample object is not in the abnormal condition;

the prediction model is used for inputting the first image and the second image into the image detection model, respectively extracting features of the first image and the second image by the image detection model, acquiring difference information between the images according to the extracted first image feature of the first image and the second image feature of the second image, predicting based on the difference information to obtain risk information of the second image, and determining the second image as a target image if the risk information meets a target condition, wherein the target image is used for indicating that the target object has the abnormal condition;

and the determining module is used for determining the shooting time of the second image as the occurrence time of the abnormal condition.

In a possible embodiment, the apparatus further comprises:

the parameter changing module is used for acquiring at least two third images, and each third image is obtained by shooting the same concerned part of the same object based on shooting equipment under the setting of different equipment parameters;

and the characteristic determining module is used for determining the image characteristics of which the difference information between the image characteristics of the at least two third images is smaller than the image characteristics of the target difference information as the image characteristics to be extracted.

In a possible embodiment, the apparatus further comprises:

the characteristic extraction module is used for carrying out characteristic extraction on the positive sample images and the negative sample images to obtain a plurality of positive sample image characteristics and a plurality of negative sample image characteristics;

the first risk information output module is used for inputting second difference information between the first positive sample image characteristic and any negative sample image characteristic into a first model, predicting by the first model based on the second difference information and outputting second risk information;

the first prediction module is used for predicting whether the negative sample image corresponding to any negative sample image feature is the image with the abnormal condition or not based on the second risk information and the initial threshold value relation;

and the adjusting module is used for adjusting the model parameters of the first model and the initial threshold value based on the difference between the prediction result and the actual result until the model parameters of the first model meet the target cutoff condition, stopping training the first model, and taking the trained first model as the image detection model.

In a possible embodiment, the apparatus further comprises:

the second risk information output module is used for inputting third difference information of the first positive sample image characteristic and the target positive sample image characteristic into the first model, predicting the third difference information by the first model based on the third difference information and outputting third risk information, wherein the target positive sample image characteristic is a positive sample image characteristic except the first positive sample image characteristic;

the second prediction module is used for predicting whether the positive sample image corresponding to the target positive sample image feature is the image with the abnormal condition or not based on the third risk information and the initial threshold value relation;

In a possible embodiment, the apparatus further comprises:

a target area determination module, configured to determine a target area in a registered image, where the registered image is an image with a resolution higher than a target threshold, and the target area is an area where the abnormal condition occurs;

a deformation registration module, configured to perform deformation registration on the multiple first sample images and the registration image, and determine a second correspondence between each pixel point in the registration image and each pixel point in the multiple first sample images, where the multiple first sample images are images of the object in the abnormal condition;

the deformation registration module is further configured to perform deformation registration on the plurality of second sample images and the registered image, and determine a third corresponding relationship between each pixel point in the registered image and each pixel point in the plurality of second sample images, where the plurality of second sample images are images of the object without the abnormal condition;

the target tendency determination module is further configured to determine the target region in the plurality of first sample images and the plurality of second sample images based on the second correspondence and the third correspondence;

a sample determination module for determining images located in the target area among the plurality of first sample images and the plurality of second sample images as the positive sample image and the negative sample image.

In one aspect, a computer device is provided that includes one or more processors and one or more memories having at least one program code stored therein, the program code being loaded and executed by the one or more processors to implement the operations performed by the image detection method.

In one aspect, a storage medium is provided, in which at least one program code is stored, the program code being loaded and executed by a processor to implement the operations performed by the image detection method.

By the image detection method provided by the embodiment of the application, the first picture shot when the target object is not in the abnormal condition is taken as a reference, the second image and the first image acquired in the target processing process are compared in real time, and whether the second image is the picture in the abnormal condition or not is determined based on the comparison result. Because the second image is obtained in real time, relevant personnel can obtain the time when the target object is abnormal in the detection of the image detection model at the first time, and make corresponding processing scheme adjustment, thereby improving the possibility and the safety of the target processing process and improving the processing effect.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic diagram of an implementation environment of an image detection method provided in an embodiment of the present application;

fig. 2 is a flowchart of an image detection method provided in an embodiment of the present application;

fig. 3 is a flowchart of an image detection method according to an embodiment of the present application;

fig. 4 is a flowchart of a detection process performed by an image detection method according to an embodiment of the present application;

FIG. 5 is a flowchart of an image detection method provided in an embodiment of the present application;

FIG. 6 is a flowchart of an image detection model training method provided in an embodiment of the present application;

fig. 7 is a schematic structural diagram of an image detection apparatus according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The terms "first," "second," and the like in this application are used for distinguishing between similar items and items that have substantially the same function or similar functionality, and it should be understood that "first," "second," and "nth" do not have any logical or temporal dependency or limitation on the number or order of execution.

The term "at least one" in this application means one or more, "a plurality" means two or more, for example, a plurality of third images means two or more third images.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

Machine Learning (ML) is a multi-domain cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge submodel to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

The radiation therapy in the embodiment of the present application is a local therapy method for treating tumor by using radiation, wherein the radiation includes α, β, gamma rays generated by radioactive isotope, and x rays, electron beams, proton beams and other particle beams generated by various types of x-ray therapy machines or accelerators.

Radiation inflammation refers to the inflammatory response of a patient to radiation damage to normal tissue during radiation therapy.

The imaging omics characteristics refer to the imaging characteristics used to characterize the tumor.

Fig. 1 is a schematic diagram of an implementation environment of an image detection method provided in an embodiment of the present application, and referring to fig. 1, the implementation environment may include a terminal 110, a shooting device 120, and a server 140.

The terminal 110 is connected to the server 110 through a wireless network or a wired network. The terminal 110 may be a smart phone, a tablet computer, a portable computer, a medical computer, a material detection computer, or the like. The terminal 110 is installed and operated with an application program supporting the image detection technology.

The photographing apparatus 120 may be an apparatus having an image photographing capability, such as a Computed Tomography (CT) apparatus or a Magnetic Resonance Imaging (MRI) apparatus, etc.; an imaging device used in processing a material, such as an Optical Microscope (OM) or a Scanning Electron Microscope (SEM); of course, the method and the device may also be a device capable of acquiring an image of a processed object in real time in other processing processes, for example, a camera used for observing a product processing condition in a numerical control machine tool, or even a camera on a smart phone.

The terminal 110 may be connected to the photographing device 120 through a wireless network or a wired network, and the terminal 110 is connected to the server 140 through a wireless network or a wired network.

Optionally, the server 140 comprises: the system comprises an access server, an image detection server and a database. The access server is used to provide access services for the terminal 110. The image detection server is used for providing background services related to image detection. The database may include a sample database, a user information database, and the like, and of course, the database may correspond to different databases based on different services provided by the server, and the image detection server may be one or more. When there are multiple image detection servers, there are at least two image detection servers for providing different services, and/or there are at least two image detection servers for providing the same service, for example, providing the same service in a load balancing manner, which is not limited in the embodiments of the present application.

The terminal 110 may be generally referred to as one of a plurality of terminals, and the embodiment of the present application is illustrated by the terminal 110.

Those skilled in the art will appreciate that the number of terminals described above may be greater or fewer. For example, the number of the terminal may be only one, or several tens or hundreds, or more, and in this case, other terminals are also included in the implementation environment. The number of terminals and the type of the device are not limited in the embodiments of the present application.

Fig. 2 is a flowchart of an image detection method provided in an embodiment of the present application, fig. 3 is a flowchart of an image detection method provided in an embodiment of the present application, and fig. 4 is a flowchart of a detection process performed by an image detection method provided in an embodiment of the present application, with reference to fig. 2, fig. 3, and fig. 4, where the method includes:

a computer device acquires a first image of a target object, the first image representing an original state of the target object before radiotherapy is administered 201.

The target object may be a human with a malignant tumor or other animals with a malignant tumor, and the kind of the target object is not limited in the embodiments of the present application. The first image may be an image captured before the target object is subjected to the radiotherapy, or may be a guide image used for guiding a radiation irradiation position in the course of performing the radiotherapy on the target object, and since the radioactive inflammation may occur only when the dose of the radiation is accumulated to a certain total amount, the acquisition time of the first image may be the first image acquired when the radiotherapy starts, so that it may be ensured that the content in the first image is an original state of the target object before the radiotherapy is performed, where the type of the first image may be a CT image or an MRI image, and the type of the first image is not limited in the embodiment of the present application.

202. The computer device acquires a second image of the target object in real time during the course of performing the radiotherapy on the target object.

The second images are images acquired in real time in the process of performing radiotherapy on the target object, the second images can be a group of temporally continuous images, each second image can correspond to a timestamp, the computer device can acquire the shooting sequence of different second images based on the timestamps, and the shooting time of the second images can also be determined.

203. The computer equipment calls an image detection model, the image detection model is obtained by training based on a positive sample image and a negative sample image obtained by radiotherapy of the target object, the positive sample image is used for indicating that the sample object generates the radioactive inflammation, and the negative sample image is used for indicating that the sample object does not generate the radioactive inflammation.

The image detection model can be obtained by training pictures acquired by a plurality of different sample objects in the radioactive treatment process as samples, has the capability of predicting whether the target object generates radioactive inflammation or not based on the difference between the first image and the second image, and can be divided into a positive sample image and a negative sample image, wherein the positive sample image is an image acquired by the sample object in the radioactive treatment process after being diagnosed with the radioactive inflammation, and the negative sample image is an image of the sample object which does not find the radioactive inflammation in the radioactive treatment process. The radiation therapy can be performed in a plurality of stages, the radioactive inflammation can be generated when the radiation dose is accumulated to a certain degree, when the negative sample image is selected, the computer device can judge the image shooting time based on the time stamps of a plurality of images, determine the image of the sample object before the radioactive inflammation is confirmed to be generated based on the image shooting time, and determine the image of the sample object before the radioactive inflammation is confirmed to be generated as the negative sample. In a possible implementation manner, the image detection model may be a binary model, such as Random Forest (RF) or Logistic Regression (LR), and the examples of the present application do not limit the type of the image detection model.

In a possible embodiment, the sample object is generally diagnosed at a time when the radioactive inflammation occurs, and the inflammation does not actually have a very obvious phenomenon when the inflammation begins to occur, in order to further improve the accuracy of the trained image detection model, the computer device may screen the negative sample image, delete the images of the target number of shooting times close to the time point when the sample object is diagnosed as having the radioactive inflammation, and screen the negative sample by using the remaining images as the negative sample, so that the accuracy of the image detection model can be improved. For example, when 10 images are acquired in total during the radiotherapy of the sample object and the related phenomena of the radioactive inflammation occurs in the sixth image, the sixth image may be determined as the image diagnosed as having the radioactive inflammation, the sixth image and 4 images after the sixth image may be used as the positive sample image, and the computer device may screen 5 images before the sixth image, for example, the first, second and third images are retained, the fourth and fifth images are deleted, and the first, second and third images are used as the negative sample images.

In a possible embodiment, the computer device may further obtain a registration image of the sample object, where the registration image is an image of the sample object with a resolution higher than a target threshold, which is captured after the sample object is diagnosed with the radioactive inflammation, and the computer device may determine a target region in the registration image, specifically, the computer device may perform deformation registration on the plurality of first sample images and the registration image, determine a second correspondence between each pixel point in the registration image and each pixel point in the plurality of first sample images, and the plurality of first sample images are images of the sample object with the radioactive inflammation. The deformation registration module is further configured to perform deformation registration on the plurality of second sample images and the registration image, and determine a third corresponding relationship between each pixel point in the registration image and each pixel point in the plurality of second sample images.

In a possible implementation, the computer device may perform image recognition on the registered image, determine a target region of the sample object where the sample object is subjected to the radioactive inflammation in the registered image, and determine coordinates of a pixel point corresponding to the target region. In particular, the computer device may determine the target region in a plurality of first sample images and a plurality of second sample images based on the second correspondence and the third correspondence, wherein the plurality of second sample images are images of the subject without undergoing a radioactive inflammation. Specifically, the target region determining module may determine a correspondence between each of pixel points in the registration image and each of the plurality of first sample images and the plurality of second sample images, and determine the target region in the plurality of first sample images and the plurality of second sample images based on the correspondence of the pixel points.

In one possible embodiment, the computer device may take images located in the target area among the plurality of first sample images and the plurality of second sample images as positive samples and negative samples. Specifically, the computer device may intercept the image corresponding to the target region in the first sample image and the second sample image, and use the intercepted image as a positive sample and a negative sample of the model training.

204. The computer equipment inputs a first image and a second image into an image detection model, feature extraction is carried out on the first image and the second image through the image detection model respectively, first difference information between the images is obtained according to extracted first image omics features of the first image and second image omics features of the second image, prediction is carried out based on the first difference information, first risk information of the second image is obtained, if the first risk information meets a target condition, the second image is determined to be a target image, and the target image is used for representing that a target object generates radioactive inflammation.

The image omics features may include first-order grayscale statistical features, morphological volume features, texture features, wavelet features, and the like. If the image detection model is an LR model, the LR model may perform feature extraction on the first image and the second image to obtain a first-order grayscale statistic value corresponding to the first image and the second image, morphological volume features of a target region in the first image and the second image, texture features of the first image and the second image, and a wavelet feature, where the first-order grayscale statistic value may be a grayscale value corresponding to each pixel point in the first image and the second image, and ranges from 0 to 255; the morphological volume feature may be pixel coordinates of the target region in the first image and the second image; the texture features can be used to represent the patterns of the regions that appear repeatedly in the first image and the second image and their arrangement rules; wavelet features may be used to represent the characteristics of the first and second images in the frequency domain. The LR model may calculate the first difference information based on a weight matrix of the model, and map a calculation result to an interval of (0,1), where the mapping result may be an image corresponding to the characteristics of the omics as an image with radiation inflammation, and specifically may be implemented based on formula (1) or formula (2).

Where P () is a probability, Y is an output result, 1 represents that the image is an image in which a radioactive inflammation occurs, 0 represents that the image is an image in which a radioactive inflammation does not occur, x is input first difference information, and w is a weight matrix of the LR model.

In a possible implementation, the computer device may input the first image and the second image into a trained image detection model, and perform feature extraction on the first image and the second image by using the image detection model, respectively, to obtain a first proteomic feature of the first image and a second proteomic feature of the second image. The image detection model may determine feature difference information between the first and second cinematographic features as first difference information between the images. The image detection model can predict based on the first difference information to obtain first risk information of the second image, and if the first risk information is larger than the target risk information, the second image is determined to be a target image, and the target image is used for indicating that the target object generates the radioactive inflammation when the second image is shot.

In one possible implementation, the computer device may segment the first image into a target number of first tiles, and perform feature extraction on the first tiles to obtain the first cinematology feature. The computer device may determine a second tile in the second image that corresponds to the first tile, perform feature extraction on the second tile to obtain a second cinematographic feature, and determine difference information between the first tile and the second tile based on the first and second cinematographic features. The image detection model may predict based on difference information between the first tile and the second tile, resulting in first risk information for the second tile. And if the first risk information of the second image block is larger than the target risk information, determining the second image block as a target image. Specifically, the computer device can randomly divide the first image into a target number of first image blocks, and perform feature extraction on each first image block to obtain a first imagery omics feature; the computer device can perform deformation registration on the first image and the second image, determine a first corresponding relation between a plurality of pixel points in the first image and a plurality of pixel points in the second image, determine a second image block corresponding to the first image block in the second image based on the first corresponding relation, and perform feature extraction on the second image block to obtain a second image omics feature. The image detection model may determine feature difference information between the first and second cinematographic features as first difference information between the first and second tiles. The image detection model can perform operation based on the first difference information to obtain first risk information of the second image block, if the first risk information is larger than the target risk information, the second image block is determined to be the target image, and if the first risk information is smaller than the target risk information, the second image block is determined not to be the target image. In this implementation, the computer device may segment the first image and the second image into a target number of first tiles and second tiles corresponding to each other, the image detection model may perform detection based on the first tiles and the second tiles, and the location in the second image where the radioactive inflammation occurs may be determined based on the detection result.

205. The computer device determines the capturing time of the target image as the occurrence time of the radioactive inflammation.

In one possible implementation, the computer device may determine a capture time of the target image based on a timestamp of the target image, the capture time of the target image being determined as an occurrence time of the radioactive inflammation. Under the implementation mode, the computer equipment can determine the occurrence time of the radioactive inflammation in real time, so that medical staff can take corresponding measures conveniently, and the effect of radioactive treatment can be improved.

All the above optional technical solutions may be combined arbitrarily to form optional embodiments of the present application, and are not described herein again.

The computer device may be connected with a shooting device, and the shooting device may be configured to collect an image, so as to output the collected image to the computer device for subsequent processing, so as to obtain the occurrence time of the radioactive inflammation. Of course, the shooting device may also be a device integrated with a computer device, which is not limited in this embodiment of the application.

By the image detection method provided by the embodiment of the application, the first picture taken when the target object does not generate the radioactive inflammation is taken as a reference, the second image and the first image acquired in the radiotherapy process are compared in real time, and whether the second image is the picture generating the radioactive inflammation or not is determined based on the comparison result. Because the second image is acquired in real time, medical staff can acquire the time of the target object generating the radioactive inflammation in the first time in the detection of the image detection model, and make corresponding treatment scheme adjustment, so that the controllability of the radioactive treatment is improved, the damage of the radioactive treatment to the normal tissue of the target object is reduced, and the effect of the radioactive treatment is improved.

Fig. 5 is a flowchart of an image detection method provided in an embodiment of the present application, and referring to fig. 5, the method includes:

501. the computer device acquires a first image of the target object, the first image representing an original state of the target object before being subjected to the target processing.

The target object may refer to a metal material to be processed, a wood material, or even a human body, if the target object is a human body, the method provided in this embodiment may be similar to the idea provided in the foregoing embodiment, and is not described herein again, and if the target object is a metal material or a wood material, the method may be performed according to step 501 and 506. If the target object is a metal material, the computer device may acquire, by using a camera device connected to the computer device, a first image of the target object, where the first image is used to represent an original state of the target object before the target object is subjected to target processing, and for the metal material, the target processing may be stretching processing of the metal material. In one possible embodiment, a camera device may be provided in the metal drawing machine, and the computer device may acquire a first image of the metal material before being drawn by the camera device; in another possible embodiment, a metal stretching device may be placed in the SEM, and the SEM may be used as a photographing device to obtain an image of the metal material during the stretching process.

502. And the computer equipment acquires a second image of the target object in real time in the process of carrying out target processing on the target object.

In a possible embodiment, the computer device may obtain, in real time, second images of the metal material during the stretching process based on a shooting device installed in the metal stretching machine, where the second images may be a group of temporally consecutive images, each of the second images may correspond to a shooting time stamp, and the computer device may obtain a time sequence of shooting different second images based on the time stamps, and may also determine the shooting time of the second images.

503. The computer equipment calls an image detection model, the image detection model is obtained by training based on a positive sample image and a negative sample image obtained in the target processing process of the sample object, the positive sample image is used for indicating that the sample object is abnormal, and the negative sample image is used for indicating that the sample object is not abnormal.

The image detection model can be obtained by training pictures acquired by a plurality of different sample objects in the process of target processing as samples, has the capability of predicting whether the target object has an abnormal condition or not based on the difference between the first image and the second image, and can be divided into a positive sample image and a negative sample image, wherein the positive sample image is an image acquired by the sample objects in the process of target processing after the sample objects have the abnormal condition, and the negative sample image is an image of the sample objects without the abnormal condition in the process of target processing. The target processing can be divided into a plurality of stages, the abnormal condition can occur when the time for implementing the target processing is accumulated to a certain extent, and when the computer device selects the negative sample image, the image shooting time can be judged based on the time stamps of the plurality of images, the image before the abnormal condition of the sample object occurs is determined based on the image shooting time, and the image before the abnormal condition of the sample object is determined as the negative sample. If the target object is a metal material and the target processing process is a stretching process, the abnormal condition may refer to that the metal material has a crack during the stretching process. In a possible implementation manner, the image detection model may be a binary model, such as Random Forest (RF) or Logistic Regression (LR), and the examples of the present application do not limit the type of the image detection model.

In a possible implementation manner, the sample object is generally already at a time when the abnormal condition is determined to occur, and the abnormal condition does not actually have a very obvious phenomenon when the abnormal condition starts to occur, in order to further improve the accuracy of the trained image detection model, the computer device may screen the negative sample image, delete the images at the time points when the target number of shooting times are close to the abnormal condition of the sample object, and screen the negative sample in such a manner, so that the accuracy of the image detection model can be improved. For example, taking the metal material as the target object as an example, if 10 images are obtained in total during the stretching process of the metal material and a crack occurs in the sixth image, the sixth image may be determined as an image in which an abnormal condition occurs, the sixth image and 4 images after the sixth image may be used as positive sample images, and the computer device may screen 5 images before the sixth image, for example, the first, second, and third images may be retained, the fourth and fifth images may be deleted, and the first, second, and third images may be used as negative sample images.

In one possible embodiment, the computer device may further acquire a registered image of the sample object, the registered image being an image of the sample object taken after the sample object is determined to have an abnormal condition with a resolution higher than a target threshold. The computer equipment can perform image recognition on the registration image, determine a target area of the sample object with abnormal conditions in the registration image, and determine pixel point coordinates corresponding to the target area. For example, if the stretching apparatus is installed in the SEM, the computer apparatus may control the SEM to acquire a registration image with a photographing parameter having a resolution higher than a target threshold, perform image recognition on the registration image, and determine a target region in the registration image where the crack occurs in the metal material.

The computer device may be configured to perform deformation registration on the plurality of first sample images and the registration image, and determine a second correspondence between each pixel point in the registration image and each pixel point in the plurality of first sample images, where the plurality of first sample images are images in which an object is abnormal. The computer device may perform deformation registration on the plurality of second sample images and the registration image, and determine a third correspondence between each pixel point in the registration image and each pixel point in the plurality of second sample images.

The computer device may further determine the target region in the plurality of first sample images and the plurality of second sample images based on the second correspondence and the third correspondence, wherein the plurality of second sample images are images in which the sample object has not been in an abnormal condition. Specifically, the computer device may determine a correspondence of the plurality of first sample images and the plurality of second sample images to each pixel point in the registration image, and determine the target region in the plurality of first sample images and the plurality of second sample images based on the correspondence of the pixel points.

The computer device may take images located at the target area among the plurality of first sample images and the plurality of second sample images as positive samples and negative samples. Specifically, the computer device may intercept the image corresponding to the target region in the first sample image and the second sample image, and use the intercepted image as a positive sample and a negative sample of the model training.

504. The computer equipment inputs the first image and the second image into an image detection model, feature extraction is respectively carried out on the first image and the second image by the image detection model, and difference information between the images is obtained according to the extracted first image feature of the first image and the extracted second image feature of the second image.

In a possible implementation manner, the computer device may input the first image and the second image into a trained image detection model, perform feature extraction on the first image and the second image by using the image detection model to obtain a first image feature of the first image and a second image feature of the second image, and obtain difference information between the images based on feature difference information between the first image feature and the second image feature.

In one possible implementation, the computer device may segment the first image into a target number of first tiles, and perform feature extraction on the first tiles to obtain the first image features. The computer device may determine a second tile corresponding to the first tile in the second image, perform feature extraction on the second tile to obtain a second image feature, and determine difference information between the first tile and the second tile based on the first image feature and the second image feature.

505. And the image detection model carries out prediction based on the difference information to obtain first risk information of the second image, and if the first risk information meets the target condition, the second image is determined as a target image which is used for representing the abnormal condition of the target object.

In a possible implementation manner, the image detection model may perform prediction based on the first difference information to obtain first risk information of the second image, and if the first risk information of the second image is greater than the target risk information, determine that the second image is a target image, where the target image is used to indicate that an abnormal condition occurs in the target object.

In one possible implementation, the image detection model performs prediction based on difference information between the first image block and the second image block to obtain first risk information of the second image block. And if the first risk information of the second image block is larger than the target risk information, determining the second image block as a target image. Specifically, the image detection model may determine feature difference information between the first image feature and the second image feature as first difference information between the first tile and the second tile. The image detection model can perform operation based on the first difference information to obtain first risk information of the second image block, if the first risk information of the second image block is larger than the target risk information, the second image block is determined to be the target image, and if the first risk information of the second image block is smaller than the target risk information, the second image block is determined not to be the target image. In this implementation, the computer device may segment the first image and the second image into a target number of first tiles and second tiles corresponding to each other, the image detection model may perform detection based on the first tiles and the second tiles, and the location of the occurrence of the abnormal situation in the second image may be determined based on the detection result.

506. The computer device determines the capturing time of the second image as the occurrence time of the abnormal situation.

In one possible implementation, the computer device may determine a shooting time of the target image based on a timestamp of the target image, and determine the shooting time of the target image as an occurrence time of the abnormal situation. Under the implementation mode, the computer equipment can determine the occurrence time of the abnormal condition in real time, so that relevant personnel can take corresponding measures conveniently, and the target processing effect can be improved.

It should be noted that, the foregoing is described by taking a metal material as an example, and the image detection method provided in the embodiment of the present application may be actually performed on different types of target objects according to actual situations, and the type of the target object is not limited in the embodiment of the present application.

Fig. 6 is a flowchart of an image detection model training method provided in an embodiment of the present application, and referring to fig. 6, the method includes:

601. the computer equipment acquires at least two third images, each third image is obtained by shooting the same concerned part of the same object based on the shooting equipment under the setting of different equipment parameters, and the image characteristics of which the difference information between the image characteristics of the at least two third images is smaller than the target difference information are determined as the image characteristics to be extracted.

The device parameters may include a plurality of parameters of the photographing device for acquiring an image, such as aperture, sensitivity, exposure, acceleration voltage, and electron beam intensity.

In one possible implementation, the computer device may control the photographing device to photograph the target object to acquire the third image. The computer device may then change the device parameters of the photographing device, and photograph the same region of interest of the target object based on the photographing device after the device parameters are changed, to obtain another third image. The computer device can perform feature extraction on the at least two third images to obtain at least two third image features. The computer device may determine difference information between the plurality of third image features, and if the difference information is smaller than the target difference information, determine an image feature corresponding to the difference information as the image feature to be extracted. Under the implementation mode, the computer equipment can obtain the image characteristics which can still keep stable after the equipment parameters are changed, and the model trained on the basis of the more stable image characteristics has higher robustness.

602. And the computer equipment performs feature extraction on the plurality of positive sample images and the plurality of negative sample images to obtain a plurality of positive sample image features and a plurality of negative sample image features.

In a possible implementation, the computer device may perform feature extraction on the plurality of positive sample images and the plurality of negative sample images based on the image features to be extracted determined in step 601, so as to obtain a plurality of positive sample image features and a plurality of negative sample image features.

603. The computer equipment inputs second difference information between the first positive sample image characteristic and any one negative sample image characteristic or third difference information between the first positive sample image characteristic and the target positive sample image characteristic into the first model, performs prediction by the first model based on the second difference information or the third difference information, and outputs second risk information, wherein the target positive sample image characteristic is a positive sample image characteristic except the first positive sample image characteristic.

The first positive sample image feature is a first sample image which is obtained according to time sequence arrangement, and the first sample image reflects the initial state of a sample object. Training a first model based on second difference information between the first positive sample image feature and any one negative sample image feature, and gradually improving the image of the first model for recognizing abnormal conditions; training the first model based on the third difference information of the first positive sample image feature and the target positive sample image feature, and gradually improving the image recognition of the first model without abnormal conditions.

604. And the computer equipment predicts whether the negative sample image corresponding to any one negative sample image characteristic is the image with the abnormal condition or not or whether the positive sample image corresponding to the target positive sample image characteristic is the image with the abnormal condition or not based on the second risk information and the initial threshold value relation.

In a possible implementation manner, if the second risk information is greater than the initial threshold, the computer device may determine that the negative sample image corresponding to the negative sample image feature is an image in which an abnormal condition occurs, or the positive sample image corresponding to the target positive sample image feature is an image in which an abnormal condition occurs. If the second risk information is smaller than the initial threshold, the computer device may determine that the negative sample image corresponding to the negative sample image feature is an image without an abnormal condition, or the positive sample image corresponding to the target positive sample image feature is an image without an abnormal condition.

605. And the computer equipment adjusts the model parameters and the initial threshold value of the first model based on the difference between the prediction result and the actual result, stops training the first model until the model parameters of the first model meet the target cutoff condition, and takes the trained first model as an image detection model.

In one possible embodiment, if the predicted result matches the actual result, the computer device may not adjust the model parameters and the initial threshold of the first model; if the predicted result does not match the actual result, the computer device may adjust the model parameters and the initial threshold of the first model. For example, the adjustment of the model parameter may be implemented by a gradient descent method or a gradient ascent method, and the adjustment of the initial threshold may be implemented based on an Area Under a working characteristic Curve (AUC) of the subject, or of course, the model parameter and the initial threshold of the first model may also be implemented by other methods, which are not limited in this embodiment. If the function value of the loss function of the first model is smaller than the objective function value, the training of the first model may be stopped, and the first model obtained at this time is used as the image detection model, or if the number of times of training of the first model reaches the objective number of times, the training of the first model is stopped, and the first model obtained at this time is used as the image detection model.

Fig. 7 is a schematic structural diagram of an image detection apparatus provided in an embodiment of the present application, and referring to fig. 7, the apparatus includes: an obtaining module 701, a model calling module 702, a prediction model 703 and a determining module 704.

An obtaining module 701, configured to obtain a first image of a target object, where the first image is used to represent an original state of the target object before target processing is performed on the target object;

the obtaining model 701 is further configured to obtain a second image of the target object in real time during a process of performing target processing on the target object;

a model calling module 702, configured to call an image detection model, where the image detection model is obtained by training a positive sample image and a negative sample image obtained in a target processing process of a sample object, the positive sample image is used to indicate that the sample object is abnormal, and the negative sample image is used to indicate that the sample object is not abnormal;

the prediction model 703 is used for inputting the first image and the second image into the image detection model, respectively extracting features of the first image and the second image by the image detection model, acquiring difference information between the images according to the extracted first image feature of the first image and the second image feature of the second image, predicting based on the difference information to obtain risk information of the second image, and determining the second image as a target image if the risk information meets a target condition, wherein the target image is used for indicating that an abnormal condition occurs in a target object;

a determining module 704, configured to determine the capturing time of the second image as the occurrence time of the abnormal situation.

In one possible embodiment, the prediction module is further configured to:

dividing the first image into a target number of first image blocks, and performing feature extraction on the first image blocks to obtain first image features;

and determining a second image block corresponding to the first image block in the second image, and performing feature extraction on the second image block to obtain a second image feature.

In a possible embodiment, the apparatus further comprises:

the position determining module is used for determining a second image block corresponding to the second image feature as a target image block; and determining the position of the target image block in the second image as the position of the target object in the abnormal condition.

In one possible embodiment, the prediction module is further configured to:

performing deformation registration on the first image and the second image, and determining a first corresponding relation between a plurality of pixel points in the first image and a plurality of pixel points in the second image;

based on the first correspondence, a second tile corresponding to the first tile is determined in the second image.

In one possible embodiment, the apparatus further comprises:

and the third image acquisition module is used for acquiring at least two third images, and each third image is obtained by shooting the same concerned part of the same object based on the shooting equipment under the setting of different equipment parameters.

In one possible embodiment, the apparatus further comprises:

and the characteristic extraction module is used for carrying out characteristic extraction on the plurality of positive sample images and the plurality of negative sample images to obtain a plurality of positive sample image characteristics and a plurality of negative sample image characteristics.

And the first risk information output module is used for inputting second difference information between the first positive sample image characteristic and any one negative sample image characteristic into the first model, predicting by the first model based on the second difference information and outputting second risk information.

And the first prediction module is used for predicting whether the negative sample image corresponding to any negative sample image feature is the image with the abnormal condition or not based on the second risk information and the initial threshold value relation.

And the adjusting module is used for adjusting the model parameters and the initial threshold of the first model based on the difference between the prediction result and the actual result until the model parameters of the first model meet the target cutoff condition, stopping training the first model, and taking the trained first model as the image detection model.

In one possible embodiment, the apparatus further comprises:

And the second risk information output module is used for inputting third difference information of the first positive sample image characteristic and the target positive sample image characteristic into the first model, predicting the third difference information by the first model based on the third difference information and outputting third risk information, wherein the target positive sample image characteristic is a positive sample image characteristic except the first positive sample image characteristic.

And the second prediction module is used for predicting whether the positive sample image corresponding to the target positive sample image feature is an image with abnormal conditions or not based on the third risk information and the initial threshold value relation.

In one possible embodiment, the apparatus further comprises:

and the target area determining module is used for determining a target area in the registration image, wherein the registration image is an image with the resolution higher than a target threshold, and the target area is an area with abnormal conditions.

And the deformation registration module is used for carrying out deformation registration on the plurality of first sample images and the registration image, determining a second corresponding relation between each pixel point in the registration image and each pixel point in the plurality of first sample images, and the plurality of first sample images are images of the object under abnormal conditions.

The deformation registration module is further configured to perform deformation registration on the plurality of second sample images and the registration image, and determine a third corresponding relationship between each pixel point in the registration image and each pixel point in the plurality of second sample images, where the plurality of second sample images are images in which the abnormal condition of the object does not occur.

The target tendency determination module is further configured to determine a target region in the plurality of first sample images and the plurality of second sample images based on the second correspondence and the third correspondence.

And the sample determining module is used for taking the images positioned in the target area in the plurality of first sample images and the plurality of second sample images as positive sample images and negative sample images.

It should be noted that: in the image detection apparatus provided in the above embodiment, when detecting an image, only the division of the above functional modules is exemplified, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the above described functions. In addition, the image detection apparatus and the image detection method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments, and are not described herein again.

By the image detection device provided by the embodiment of the application, the first picture shot when the target object has no abnormal condition is taken as a reference, the second image and the first image acquired in the target processing process are compared in real time, and whether the second image is the picture with the abnormal condition or not is determined based on the comparison result. Because the second image is obtained in real time, relevant personnel can obtain the time when the target object is abnormal in the detection of the image detection model at the first time, and make corresponding processing scheme adjustment, thereby improving the possibility and the safety of the target processing process and improving the processing effect.

Fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present application. The computer device 800 may be: a smartphone, a tablet, a laptop, or a desktop computer. Computer device 800 may also be referred to by other names such as user device, portable computer device, laptop computer device, desktop computer device, and so forth.

Generally, the computer device 800 includes: one or more processors 801 and one or more memories 802.

The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so forth. The processor 801 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor, where the main processor is a processor for processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 801 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 802 may include one or more storage media, which may be non-transitory. Memory 802 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory storage medium in the memory 802 is used to store at least one program code for execution by the processor 801 to implement the image detection methods provided by the method embodiments herein.

In some embodiments, the computer device 800 may further optionally include: a peripheral interface 803 and at least one peripheral. The processor 801, memory 802 and peripheral interface 803 may be connected by bus or signal lines. Various peripheral devices may be connected to peripheral interface 803 by a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 804, a display 805, a camera 806, an audio circuit 807, a positioning component 808, and a power supply 809.

The peripheral interface 803 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 801 and the memory 802. In some embodiments, the processor 801, memory 802, and peripheral interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral interface 803 may be implemented on separate chips or circuit boards, which are not limited by this embodiment.

The Radio Frequency circuit 804 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 804 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 804 converts an electrical signal into an electromagnetic signal to be transmitted, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuit 804 may communicate with other computer devices via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: metropolitan area networks, various generation mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency circuit 804 may further include NFC (Near Field Communication) related circuits, which are not limited in this application.

The display screen 805 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display 805 is a touch display, the display 805 also has the ability to capture touch signals on or above the surface of the display 805. The touch signal may be input to the processor 801 as a control signal for processing. At this point, the display 805 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 805 may be one, providing the front panel of the computer device 800; in other embodiments, the display 805 may be at least two, each disposed on a different surface of the computer device 800 or in a folded design; in still other embodiments, the display 805 may be a flexible display, disposed on a curved surface or on a folded surface of the computer device 800. Even further, the display 805 may be arranged in a non-rectangular irregular pattern, i.e., a shaped screen. The Display 805 can be made of LCD (Liquid Crystal Display), OLED (Organic Light-emitting diode), and the like.

The camera assembly 806 is used to capture images or video. Optionally, camera assembly 806 includes a front camera and a rear camera. Generally, a front camera is disposed on a front panel of a computer apparatus, and a rear camera is disposed on a rear surface of the computer apparatus. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 806 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

The audio circuit 807 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 801 for processing or inputting the electric signals to the radio frequency circuit 804 to realize voice communication. For stereo capture or noise reduction purposes, the microphones may be multiple and located at different locations on the computer device 800. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 801 or the radio frequency circuit 804 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, the audio circuitry 807 may also include a headphone jack.

The Location component 808 is used to locate the current geographic Location of the computer device 800 to implement navigation or LBS (Location Based Service). The positioning component 808 may be a positioning component based on the GPS (global positioning System) in the united states, the beidou System in china, the graves System in russia, or the galileo System in the european union.

A power supply 809 is used to power the various components in the computer device 800. The power supply 809 can be ac, dc, disposable or rechargeable. When the power source 809 comprises a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, the computer device 800 also includes one or more sensors 810. The one or more sensors 810 include, but are not limited to: acceleration sensor 811, gyro sensor 812, pressure sensor 813, fingerprint sensor 814, optical sensor 815 and proximity sensor 816.

The acceleration sensor 811 may detect the magnitude of acceleration in three coordinate axes of a coordinate system established with the computer apparatus 800. For example, the acceleration sensor 811 may be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 801 may control the display 805 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 811. The acceleration sensor 811 may also be used for acquisition of motion data of a game or a user.

The gyro sensor 812 may detect a body direction and a rotation angle of the computer device 800, and the gyro sensor 812 may cooperate with the acceleration sensor 811 to acquire a 3D motion of the user with respect to the computer device 800. From the data collected by the gyro sensor 812, the processor 801 may implement the following functions: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.

Pressure sensors 813 may be disposed on the side bezel of computer device 800 and/or underneath display screen 805. When the pressure sensor 813 is arranged on the side frame of the computer device 800, the holding signal of the user to the computer device 800 can be detected, and the processor 801 performs left-right hand identification or shortcut operation according to the holding signal collected by the pressure sensor 813. When the pressure sensor 813 is disposed at a lower layer of the display screen 805, the processor 801 controls the operability control on the UI interface according to the pressure operation of the user on the display screen 805. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.

The fingerprint sensor 814 is used for collecting a fingerprint of the user, and the processor 801 identifies the identity of the user according to the fingerprint collected by the fingerprint sensor 814, or the fingerprint sensor 814 identifies the identity of the user according to the collected fingerprint. Upon identifying that the user's identity is a trusted identity, the processor 801 authorizes the user to perform relevant sensitive operations including unlocking a screen, viewing encrypted information, downloading software, paying for and changing settings, etc. Fingerprint sensor 814 may be disposed on the front, back, or side of computer device 800. When a physical key or vendor Logo is provided on the computer device 800, the fingerprint sensor 814 may be integrated with the physical key or vendor Logo.

The optical sensor 815 is used to collect the ambient light intensity. In one embodiment, processor 801 may control the display brightness of display 805 based on the ambient light intensity collected by optical sensor 815. Specifically, when the ambient light intensity is high, the display brightness of the display screen 805 is increased; when the ambient light intensity is low, the display brightness of the display 805 is reduced. In another embodiment, the processor 801 may also dynamically adjust the shooting parameters of the camera assembly 806 based on the ambient light intensity collected by the optical sensor 815.

A proximity sensor 816, also known as a distance sensor, is typically provided on the front panel of the computer device 800. The proximity sensor 816 is used to capture the distance between the user and the front of the computer device 800. In one embodiment, the processor 801 controls the display 805 to switch from the bright screen state to the dark screen state when the proximity sensor 816 detects that the distance between the user and the front face of the computer device 800 is gradually reduced; when the proximity sensor 816 detects that the distance between the user and the front of the computer device 800 is gradually increasing, the display screen 805 is controlled by the processor 801 to switch from a breath-screen state to a bright-screen state.

Those skilled in the art will appreciate that the configuration illustrated in FIG. 8 is not intended to be limiting of the computer device 800 and may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components may be employed.

In an exemplary embodiment, there is also provided a storage medium, such as a memory, including program code executable by a processor to perform the image detection method in the above-described embodiments. For example, the storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

It will be understood by those skilled in the art that all or part of the steps of implementing the above embodiments may be implemented by hardware, or may be implemented by hardware associated with program code, and the program may be stored in a storage medium, such as a read-only memory, a magnetic disk or an optical disk.

The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. An image detection method, characterized in that the method comprises:

inputting the first image and the second image into the image detection model, respectively extracting features of the first image and the second image by the image detection model, acquiring difference information between the images according to the extracted first image feature of the first image and the second image feature of the second image, predicting based on the difference information to obtain first risk information of the second image, and determining the second image as a target image if the first risk information meets a target condition, wherein the target image is used for indicating that the target object has the abnormal condition;

2. The method of claim 1, wherein the performing, by the image detection model, feature extraction on the first image and the second image respectively comprises:

and determining a second image block corresponding to the first image block in the second image, and performing feature extraction on the second image block to obtain the second image feature.

3. The method of claim 2, wherein after determining the second image as the target image, the method further comprises:

determining a second image block corresponding to the second image feature as a target image block;

and determining the position of the target image block in the second image as the position of the target object in the abnormal condition.

4. The method of claim 2, wherein determining a second tile in the second image that corresponds to the first tile comprises:

determining a second tile in the second image that corresponds to the first tile based on the first correspondence.

5. The method of claim 1, wherein before the feature extraction of the first and second images by the image detection model, respectively, the method further comprises:

acquiring at least two third images, wherein each third image is obtained by shooting the same concerned part of the same object based on shooting equipment under the setting of different equipment parameters;

and determining the image characteristics of the at least two third images, of which the difference information is smaller than that of the target difference information, as the image characteristics to be extracted.

6. The method of claim 1, wherein prior to invoking the image detection model, the method further comprises:

performing feature extraction on the plurality of positive sample images and the plurality of negative sample images to obtain a plurality of positive sample image features and a plurality of negative sample image features;

inputting second difference information between the first positive sample image feature and any negative sample image feature into a first model, predicting by the first model based on the second difference information, and outputting second risk information;

predicting whether the negative sample image corresponding to any negative sample image feature is the image with the abnormal condition or not based on the second risk information and the initial threshold value relation;

and adjusting the model parameters of the first model and the initial threshold value based on the difference between the prediction result and the actual result until the model parameters of the first model meet the target cutoff condition, stopping training the first model, and taking the trained first model as the image detection model.

7. The method of claim 1, wherein prior to invoking the image detection model, the method further comprises:

inputting third difference information of the first positive sample image feature and the target positive sample image feature into a first model, predicting by the first model based on the third difference information, and outputting third risk information, wherein the target positive sample image feature is a positive sample image feature except the first positive sample image feature;

predicting whether the positive sample image corresponding to the target positive sample image feature is the image with the abnormal condition or not based on the third risk information and the initial threshold value relation;

8. The method of any one of claims 6 or 7, wherein prior to performing feature extraction on the plurality of positive sample images and the plurality of negative sample images, the method further comprises:

determining a target area in a registration image, wherein the registration image is an image with the resolution higher than a target threshold value, and the target area is an area where the abnormal condition occurs;

performing deformation registration on the plurality of first sample images and the registration image, and determining a second corresponding relationship between each pixel point in the registration image and each pixel point in the plurality of first sample images, wherein the plurality of first sample images are images of the object under the abnormal condition;

performing deformation registration on a plurality of second sample images and the registered image, and determining a third corresponding relation between each pixel point in the registered image and each pixel point in the plurality of second sample images, wherein the plurality of second sample images are images of the object without the abnormal condition;

determining the target region in the plurality of first sample images and the plurality of second sample images based on the second correspondence and the third correspondence;

taking images of the plurality of first sample images and the plurality of second sample images located in the target area as the positive sample image and the negative sample image.

9. The method of claim 1, wherein the image detection model is a binary model.

10. An image detection apparatus, characterized in that the apparatus comprises:

11. The apparatus of claim 10, wherein the prediction module is further configured to:

12. The apparatus of claim 11, further comprising:

13. The apparatus of claim 11, wherein the prediction module is further configured to:

14. A computer device comprising one or more processors and one or more memories having at least one program code stored therein, the program code being loaded and executed by the one or more processors to perform operations performed by the image detection method of any one of claims 1 to 9.

15. A storage medium having stored therein at least one program code, the program code being loaded into and executed by a processor to perform operations performed by an image detection method according to any one of claims 1 to 9.