CN117495850B

CN117495850B - Method, device and equipment for detecting abnormal points of image

Info

Publication number: CN117495850B
Application number: CN202311841616.7A
Authority: CN
Inventors: 刘雨菡; 王子铭; 曹华钊; 聂婧; 周光尧; 胡玉新
Original assignee: Aerospace Information Research Institute of CAS
Current assignee: Aerospace Information Research Institute of CAS
Priority date: 2023-12-29
Filing date: 2023-12-29
Publication date: 2024-03-19
Anticipated expiration: 2043-12-29
Also published as: CN117495850A

Abstract

The invention provides an image abnormal point detection method which can be applied to the technical field of image processing. The method comprises the following steps: detecting a target image by using a target detection model to obtain a target area; sliding window processing is carried out on the target area to obtain a plurality of first areas; binarizing the target image according to the gray value of the pixel point included in each first area to obtain a plurality of binary mask images; respectively carrying out multi-scale division on the plurality of first areas to obtain a plurality of target subareas of each first area in the plurality of first areas; calculating the characteristic value corresponding to each target subarea according to the gray values of the pixel points included in the target subareas of the first areas respectively to obtain a characteristic value group; and calculating a plurality of first abnormal pixel point groups of the plurality of binary mask images and a second abnormal pixel point group of the characteristic value group to obtain target abnormal pixel points of the target image.

Description

Method, device and equipment for detecting abnormal points of image

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, and a device for detecting an abnormal point of an image.

Background

The infrared imaging detection system has the advantages of observation, flexible deployment and the like in all days, and can play an important role in the fields of security, environmental monitoring, aerospace and the like. Specifically, the infrared imaging detection system can be used for acquiring the state change information of the ground object, and weather early warning such as early warning of geological disasters can be performed through the state change information of the ground object, and security alarm and industrial management prompt can be performed. The key link of the infrared imaging detection system is detection of infrared image abnormal points.

At present, the more commonly used detection methods for the abnormal points of the infrared image mainly comprise a detection method based on threshold segmentation, a detection method based on filtering, a detection method based on high-order statistical characteristics, a detection method based on an image distribution model and a detection method based on a feature map. When the infrared image comprises a complex ground object and a false alarm source background with the same high radiation characteristic as the complex ground object, the detection accuracy of the detection method based on threshold segmentation is lower. The detection method based on filtering and the detection method based on high-order statistical characteristics are both dependent on parameter setting, so that the scene adaptability is poor; the detection method based on the image distribution model needs to model a neighborhood region of a ground object region in an infrared image, the detection method based on the feature map needs to perform feature calculation on the neighborhood region of the ground object region in the infrared image, and when the infrared image has higher resolution, the detection method based on the image distribution model and the detection method based on the feature map need to perform pretreatment such as blocking and restoration and post-treatment on the infrared image, so that the increase of calculation complexity and the increase of calculation resources are caused.

Disclosure of Invention

In view of the above problems, the present invention provides an image outlier detection method, apparatus, and device.

According to a first aspect of the present invention, there is provided an image outlier detection method comprising:

detecting an acquired target image by using a target detection model to obtain a target area in the target image, wherein the target detection model is obtained by training an initial target detection model according to a training sample, the training sample comprises a plurality of images related to the sample image and a plurality of labels, and the labels represent the types of the sample areas in the image; carrying out sliding window treatment on the target area to obtain a plurality of first areas; respectively carrying out binarization processing on the target image according to the gray value of the pixel point included in each first area to obtain a plurality of binary mask images of the target image, wherein the pixel point with the gray value of 1 in the binary mask images is a first abnormal pixel point; aiming at the ith first area, carrying out multi-scale division on the ith first area to obtain a plurality of target subareas of the ith first area, wherein the types of the target subareas comprise inner neighborhoods or outer neighborhoods, the gray values of all pixel points in the inner neighborhoods are different, the gray values of all pixel points in the outer neighborhoods are the same, and i is a positive integer; calculating characteristic values corresponding to each target sub-region according to gray values of pixel points included in the target sub-regions of the first regions respectively to obtain a characteristic value group of the target region, wherein the pixel points corresponding to the characteristic values in the characteristic value group are second abnormal pixel points; and calculating the first abnormal pixel point group and the second abnormal pixel point group of the characteristic value group of each of the plurality of binary mask images to obtain a target abnormal pixel point in the target image.

According to an embodiment of the present invention, for an ith first region, performing multi-scale division on the ith first region to obtain a plurality of target sub-regions of the ith first region, including: dividing an ith first region on a first scale to obtain a plurality of first sub-regions; dividing each first subarea in the plurality of first subareas on a second scale to obtain a plurality of target subareas of the ith first subarea, wherein the second scale is smaller than the first scale.

According to an embodiment of the present invention, calculating, according to gray values of pixel points included in a plurality of target sub-areas of a plurality of first areas, feature values corresponding to each of the target sub-areas, respectively, to obtain a feature value set of the target area includes: for the j-th first region, under the condition that the target sub-region is determined to be an inner neighborhood, determining a plurality of outer neighborhoods of the target sub-region, wherein j is a positive integer; calculating the maximum value of the gray values of the pixel points included in the plurality of outer neighborhoods to obtain the maximum gray value of the plurality of outer neighborhoods; calculating the characteristic value of the pixel point included in the target sub-region according to the maximum gray value to obtain the characteristic value of the target sub-region; combining the characteristic values of each target subarea in the target subareas to obtain a characteristic value group of a j-th first area, wherein the type of the target subarea is an inner neighborhood; and combining the characteristic value sets of the plurality of first areas to obtain the characteristic value set of the target area.

According to the embodiment of the invention, according to the maximum gray value, calculating the characteristic value of the pixel point included in the target subarea to obtain the characteristic value of the target subarea, including: calculating the difference value between each pixel point in the pixel points included in the target subarea and the maximum gray value to obtain a difference value group of the target subarea; and calculating the maximum value of the difference value group to obtain the characteristic value of the target subarea.

According to an embodiment of the present invention, combining the feature values of each of the plurality of target subregions to obtain a feature value set of the j-th first region includes: sequencing each target subarea in the plurality of target subareas to obtain a first sequencing result; according to the first sorting result, sorting the characteristic value of each target subarea in the plurality of target subareas to obtain a second sorting result; and combining the characteristic values in the second sequencing result to obtain a characteristic value group of the j first region.

According to an embodiment of the present invention, combining the feature value sets of the plurality of first regions to obtain the feature value set of the target region includes: sequencing each first region in the plurality of first regions to obtain a third sequencing result; according to the third sorting result, sorting the characteristic value groups of each first region in the plurality of first regions to obtain a fourth sorting result; and combining the characteristic value groups in the fourth sorting result to obtain the characteristic value groups of the target area.

According to an embodiment of the present invention, binarizing a target image according to gray values of pixel points included in each first area, to obtain a plurality of binary mask images of the target image, including: for a kth first area, calculating a gray value mean value and a gray value standard deviation of pixel points included in the kth first area, wherein k is a positive integer;

according to the gray value mean value and the gray value standard deviation, calculating a plurality of gray threshold values of the kth first area to obtain a gray threshold value group of the kth first area; and respectively carrying out binarization processing on the target image by using the gray threshold value groups of the first areas to obtain a plurality of binary mask images of the target image.

According to an embodiment of the present invention, a method for calculating a first abnormal pixel group and a second abnormal pixel group of a feature value group of each of a plurality of binary mask images to obtain a target abnormal pixel in a target image includes: respectively carrying out intersection operation on a plurality of first abnormal pixel point groups of a plurality of binary mask images and a second abnormal pixel point group corresponding to the characteristic value group to obtain a plurality of intersection sets; and performing union operation on the plurality of intersection sets to obtain a plurality of target abnormal pixel points in the target image.

A second aspect of the present invention provides an image outlier detection apparatus comprising:

the detection module is used for detecting the acquired target image by utilizing a target detection model to obtain a target area in the target image, wherein the target detection model is obtained by training an initial target detection model according to a training sample, the training sample comprises a plurality of images related to the sample image and a plurality of labels, and the labels represent the types of the sample areas in the image.

And the sliding window processing module is used for carrying out sliding window processing on the target area to obtain a plurality of first areas.

And the binarization processing module is used for respectively carrying out binarization processing on the target image according to the gray value of the pixel point included in each first area to obtain a plurality of binary mask images of the target image, wherein the pixel point with the gray value of 1 in the binary mask images is a first abnormal pixel point.

The multi-scale division module is used for carrying out multi-scale division on the ith first area to obtain a plurality of target subareas of the ith first area, wherein the types of the target subareas comprise an inner neighborhood or an outer neighborhood, the gray values of each pixel point in the inner neighborhood are different, the gray values of each pixel point in the outer neighborhood are identical, and i is a positive integer.

The first calculation module is used for calculating the characteristic value corresponding to each target subarea according to the gray values of the pixel points included in the target subareas of the first areas respectively to obtain a characteristic value group of the target area, wherein the pixel point corresponding to the characteristic value in the characteristic value group is a second abnormal pixel point.

The second calculation module is used for calculating the first abnormal pixel point group and the second abnormal pixel point group of the characteristic value group of each of the plurality of binary mask images to obtain target abnormal pixel points in the target image.

A third aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the above-described image outlier detection method.

According to the embodiment of the invention, the acquired target image is detected through the target detection model, the target image is divided into the target areas, and the abnormal pixel points in the target areas are detected, so that the target abnormal pixel points of the target image are obtained, and the adaptability of detecting the abnormal points of the image is improved. The target area is divided into the target subareas with smaller scales, the corresponding characteristic values of the pixel points in the target subareas are calculated, the characteristic values of the target areas are obtained according to the characteristic values of the target subareas, the timeliness of detecting abnormal points of the image is improved, a great amount of requirements on calculation resources are further reduced, parameters are not required to be set when the characteristic values are calculated, and the robustness of detecting the abnormal points of the image is improved to a certain extent. By calculating the first abnormal pixel point group and the second abnormal pixel point group of the characteristic value group of each of the plurality of binary mask images, more accurate target abnormal pixel points are obtained, and the detection precision of the abnormal point detection of the images is improved.

Drawings

The foregoing and other objects, features and advantages of the invention will be apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

FIG. 1 schematically illustrates an application scenario diagram of an image outlier detection method, apparatus and device according to an embodiment of the invention;

fig. 2 schematically shows a flowchart of an image outlier detection method according to an embodiment of the present invention;

FIG. 3 schematically illustrates a flow chart of multi-scale division of an ith first region to obtain multiple target sub-regions of the ith first region, in accordance with an embodiment of the invention;

fig. 4 schematically illustrates a flowchart for obtaining a set of characteristic values of a target region according to gray values of pixels included in a plurality of target sub-regions of each of a plurality of first regions according to an embodiment of the present invention;

FIG. 5 schematically illustrates a flow chart for deriving a characteristic value of a target sub-region from a maximum gray value, according to an embodiment of the invention;

FIG. 6 schematically illustrates a flowchart of binarizing a target image to obtain a plurality of binary mask images of the target image, according to an embodiment of the present invention;

FIG. 7 schematically illustrates a diagram of a multi-scale division result of a first region according to a specific embodiment of the present invention;

FIG. 8 schematically illustrates an example schematic diagram of image outlier detection according to an embodiment of the present invention;

fig. 9 schematically shows a block diagram of the structure of an image outlier detection apparatus according to an embodiment of the present invention; and

fig. 10 schematically shows a block diagram of an electronic device adapted to implement the method of detecting outliers in an image according to an embodiment of the invention.

Detailed Description

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. It should be understood that the description is only illustrative and is not intended to limit the scope of the invention. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the invention. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the present invention.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.

Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

In the technical scheme of the invention, the related user information (including but not limited to user personal information, user image information, user equipment information, such as position information and the like) and data (including but not limited to data for analysis, stored data, displayed data and the like) are information and data authorized by a user or fully authorized by all parties, and the related data are collected, stored, used, processed, transmitted, provided, invented, applied and the like, and all processed according to the related laws and regulations and standards of related countries and regions, necessary security measures are adopted, the public welfare is not violated, and corresponding operation inlets are provided for the user to select authorization or rejection.

The embodiment of the invention provides an image outlier detection method, which utilizes a target detection model to detect an acquired target image to obtain a target area in the target image, wherein the target detection model is obtained by training an initial target detection model according to a training sample, the training sample comprises a plurality of images and a plurality of labels related to a sample image, and the labels represent the types of the sample area in the image; carrying out sliding window processing on the target area to obtain a plurality of first areas of the target area; respectively carrying out binarization processing on the target image according to the gray value of the pixel point included in each first area in the plurality of first areas to obtain a plurality of binary mask images of the target image, wherein the pixel point with the gray value of 1 in the binary mask images represents a first abnormal pixel point; aiming at the ith first area, carrying out multi-scale division on the ith first area to obtain a plurality of target subareas of the ith first area, wherein the types of the target subareas comprise inner neighborhoods or outer neighborhoods, gray values of pixel points in the inner neighborhoods are different, gray values of pixel points in the outer neighborhoods are the same, and i is a positive integer; calculating characteristic values corresponding to a plurality of target subregions of each first region in the plurality of first regions according to gray values of pixel points included in the plurality of target subregions of each first region in the plurality of first regions respectively to obtain a characteristic value group of the target region, wherein the pixel points corresponding to the characteristic values in the characteristic value group are second abnormal pixel points; and calculating a plurality of first abnormal pixel point groups of the plurality of binary mask images and a second abnormal pixel point group of the characteristic value group to obtain target abnormal pixel points in the target image.

Fig. 1 schematically shows an application scenario diagram of image outlier detection according to an embodiment of the present invention.

As shown in fig. 1, the application scenario according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium used to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 via the network 104 using the first terminal device 101, the second terminal device 102, the third terminal device 103, to receive or send messages etc. Various communication client applications, such as a shopping class application, a web browser application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only) may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103.

The first terminal device 101, the second terminal device 102, the third terminal device 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.

The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.

It should be noted that the method for detecting abnormal points in an image according to the embodiment of the present invention may be generally performed by the server 105. Accordingly, the image outlier detecting apparatus provided by the embodiment of the present invention may be generally disposed in the server 105. The method for detecting abnormal points of an image provided by the embodiment of the present invention may also be performed by a server or a server cluster which is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103 and/or the server 105. Accordingly, the image outlier detecting apparatus provided by the embodiment of the present invention may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105.

It should be understood that the number of first terminal devices, second terminal devices, third terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of first, second, third, network and server terminals, as desired for implementation.

The method for detecting abnormal points in an image according to an embodiment of the present invention will be described in detail with reference to fig. 2 to 8 based on the scenario described in fig. 1.

Fig. 2 schematically shows a flowchart of an image outlier detection method according to an embodiment of the present invention.

As shown in FIG. 2, the method includes operations S210-S260.

In operation S210, the obtained target image is detected by using a target detection model to obtain a target region in the target image, where the target detection model is obtained by training an initial target detection model according to a training sample, the training sample includes a plurality of images related to the sample image and a plurality of labels, and the labels represent types of the sample region in the image.

In operation S220, a sliding window process is performed on the target area to obtain a plurality of first areas.

In operation S230, binarization processing is performed on the target image according to the gray value of the pixel point included in each first region, so as to obtain a plurality of binary mask images of the target image, where the pixel point with the gray value of 1 in the binary mask images is the first abnormal pixel point.

In operation S240, for the ith first region, the ith first region is subjected to multi-scale division to obtain multiple target subregions of the ith first region, where the types of the target subregions include an inner neighborhood or an outer neighborhood, gray values of each pixel point in the inner neighborhood are different, gray values of each pixel point in the outer neighborhood are the same, and i is a positive integer.

In operation S250, the feature values corresponding to each target sub-region are calculated according to the gray values of the pixel points included in the target sub-regions of the first regions, respectively, to obtain a feature value set of the target region, where the pixel point corresponding to the feature value in the feature value set is a second abnormal pixel point.

In operation S260, the first abnormal pixel point group and the second abnormal pixel point group of the feature value group of each of the plurality of binary mask images are calculated to obtain the target abnormal pixel point in the target image.

According to an embodiment of the present invention, the sample area is an area comprising a target object, which may be any type of object, such as any ground object type of object, the sample area may be represented by a geometric shape. The sample area where the target object is located is represented by a square, for example, and the sample area where the target object is located is represented by an oval.

According to the embodiment of the invention, the sample image can be an infrared remote sensing image, and the sample image can comprise one type of sample area or multiple types of sample areas. For example, a sample region of the category kitten may be included in the sample image. Sample images may include sample regions of the class kittens and sample regions of the class puppies.

According to an embodiment of the present invention, the category of the target area may be a category of an arbitrary sample area. The number of target areas may be any positive integer, and the size of the target areas is smaller than the size of the target image.

According to the embodiment of the invention, the target detection model is used for detecting the target area of the target object in the target image to obtain a detection result, and the detection result comprises the size of the target area and the position of the target area.

According to embodiments of the invention, the sliding window size may be less than or equal to the size of the target area.

According to an embodiment of the present invention, each of the plurality of first regions may have the same size or may have different sizes.

According to the embodiment of the invention, sliding window scanning is carried out on the target area according to the preset step length and the preset sliding window size, so that a plurality of first areas of the target area are obtained.

According to the embodiment of the invention, for each first region in the plurality of first regions, the gray value mean and the gray value variance of the pixel points included in each first region are calculated, and the gray value mean and the gray value variance of each first region are combined in different proportions to obtain a plurality of gray threshold values of each first region. And respectively carrying out binarization processing on the target image according to a plurality of gray threshold values of each of the plurality of first areas to obtain a plurality of binary mask images of the target image.

According to an embodiment of the present invention, the multi-scale division is a process of dividing the first region into a plurality of target sub-regions having the same size by a plurality of divisions of different preset sizes. Specifically, the first region may be divided into a plurality of first target sub-regions of a first preset size by dividing the first preset size, and a plurality of second target sub-regions of each of the plurality of first target sub-regions may be obtained by dividing each of the plurality of first target sub-regions by a second preset size, the plurality of second target sub-regions having the same second preset size.

According to an embodiment of the present invention, for example, the size of the target area is 400400, the multiscale division into two scales of 200 +.>200、100/>100, then by dividing the first scale, 4 dimensions 200 +.>200 by 200 +.>200, and obtaining 16 zones with the size of 100 +.>100.

According to an embodiment of the present invention, the inner neighborhood and the outer neighborhood have the same size, and the inner neighborhood and the outer neighborhood may include the same number of pixels.

According to the embodiment of the invention, when the target sub-region is an inner neighborhood, the coordinates of the pixel point and the last preset size can be calculated according to the preset size in dividing and the pixel point of the target sub-region, so as to obtain the region of the last preset size corresponding to the target sub-region. And traversing the gray values of the pixel points in the outer neighborhood included in the previous preset size area to obtain the maximum gray value of the outer neighborhood included in the previous preset size area. And calculating the characteristic value of the pixel point included in the target sub-region according to the maximum gray value to obtain the characteristic value of the target sub-region.

For example, the preset dimensions in dividing include: 2 pixel points2 pixels, 4 pixels->4 pixel points, wherein the target sub-region is an inner neighborhood, and the size of the target sub-region is 2 pixel points +.>2 pixels, the last preset size corresponding to the target sub-region is 4 pixels +.>4 pixels. Coordinates of the pixel points in the target subarea are (3, 3), (3, 4), (4, 3) and (4, 4), the pixel point with the coordinate of (4, 4) and the last preset size are calculated, and the vertex points are (1, 1), (1, 4), (4, 1) and (4, 4), and the size is 4 pixel points>The last preset size area of 4 pixels.

According to the embodiment of the invention, the characteristic value group of the target area is obtained by traversing all the inner neighborhoods in the plurality of target subareas and calculating the characteristic value of each inner neighborhood and combining the characteristic values of all the inner neighborhoods according to a certain arrangement mode. For example, a sorting result is obtained through the position relation of the first pixel point of the inner neighborhood, and the characteristic values of all the inner neighborhood are sorted according to the sorting result, so that the characteristic value group of the target area is obtained.

According to the embodiment of the invention, the gray value of the first abnormal pixel point is larger than the gray value of the pixel point with a first preset proportion in the target area.

According to the embodiment of the invention, the gray value of the second abnormal pixel point is larger than the gray value of the second preset proportion pixel points in the target subarea where the second abnormal pixel point is located.

According to the embodiment of the invention, the gray value of the target abnormal pixel point is larger than the gray value of the third preset proportion pixel point in the target area.

The first preset proportion, the second preset proportion and the third preset proportion may be configured according to actual service requirements, which is not limited herein. For example, the first preset proportion, the second preset proportion, and the third preset proportion may all be set to 90%. Alternatively, the first preset ratio may be set to 85%, the second preset ratio may be set to 85%, and the third preset ratio may be set to 95% each.

According to the embodiment of the invention, the first abnormal pixel point group may include a plurality of first abnormal pixel points and positions corresponding to the plurality of first abnormal pixel points, and the second abnormal pixel point group may include a plurality of second abnormal pixel points and positions corresponding to the plurality of second abnormal pixel points.

According to the embodiment of the invention, the target abnormal pixel point in the target image is obtained by calculating the abnormal pixel point shared by the first abnormal pixel point group and the second abnormal pixel point group.

Fig. 3 schematically shows a flowchart of multi-scale division of an ith first region to obtain multiple target sub-regions of the ith first region according to an embodiment of the present invention.

In this embodiment, as shown in fig. 3, the i-th first region is subjected to multi-scale division, and obtaining the multiple target sub-regions of the i-th first region includes operations S310 to S320.

In operation S310, the ith first region is divided on the first scale to obtain a plurality of first sub-regions.

According to an embodiment of the present invention, the first dimension may be smaller than the dimension of the first region, the length of the first region may be a positive integer multiple of the length of the first dimension, and the width of the first region may be a positive integer multiple of the width of the first dimension. The length and width of the first dimension may be the same.

According to the embodiment of the invention, the first area can be traversed through a preset sequence. For example, the predetermined sequence may be left to right, top to bottom. In the traversing process, the traversing scale is a first scale, the traversing is performed on the first area in a left-to-right sequence, the step length can be the width of the first scale, the traversing is performed on the first area in a top-to-bottom sequence, the step length can be the length of the first scale, and a plurality of first sub-areas are obtained by taking each traversed area as the first sub-area.

In operation S320, each first sub-region of the plurality of first sub-regions is divided on a second scale, so as to obtain a plurality of target sub-regions of the ith first region, where the second scale is smaller than the first scale.

According to an embodiment of the present invention, the length of the first scale may be a positive integer multiple of the length of the second scale, and the width of the first scale may be a positive integer multiple of the width of the second scale. The length and width of the second dimension may be the same.

According to the embodiment of the invention, the first sub-region can be traversed from left to right in sequence from top to bottom, in the traversing process, the traversing scale is the second scale, the first sub-region is traversed from left to right in sequence, the step length is the width of the second scale, the first sub-region is traversed from top to bottom in sequence, the step length is the length of the second scale, and a plurality of target sub-regions are obtained by taking each traversed region as a target sub-region.

According to the embodiment of the invention, each first subarea in the plurality of first subareas is divided into a plurality of target subareas according to the traversing scale by traversing the plurality of first subareas respectively.

Fig. 4 schematically shows a flowchart for obtaining a set of characteristic values of a target region according to gray values of pixel points included in a plurality of target sub-regions of each of a plurality of first regions according to an embodiment of the present invention.

In this embodiment, as shown in fig. 4, according to the gray values of the pixel points included in the plurality of target sub-areas of each of the plurality of first areas, obtaining the set of characteristic values of the target area includes operations S410 to S450.

In operation S410, for the j-th first region, in the case where the target sub-region is determined to be an inner neighborhood, a plurality of outer neighborhoods of the target sub-region are determined, where j is a positive integer.

According to the embodiment of the invention, the size of the target sub-region is obtained by traversing the pixel points of the target sub-region. And determining a plurality of outer neighborhoods of the target subarea according to the size of the target subarea and the positions of the pixel points in the target subarea. Specifically, according to a certain coordinate of a certain determined pixel point in the target sub-region, an addition or subtraction operation is performed on an abscissa of the pixel point and a width of the target sub-region, and an addition or subtraction operation is performed on an ordinate of the pixel point and a length of the target sub-region, so as to obtain coordinates of a plurality of pixel points. And combining the plurality of pixel points according to the size of the target sub-region to obtain a plurality of outer neighborhoods of the target sub-region.

In operation S420, a maximum value of gray values of pixel points included in the plurality of outer neighborhoods is calculated to obtain a maximum gray value of the plurality of outer neighborhoods.

According to the embodiment of the invention, the maximum gray value of each of the plurality of outer neighbors is obtained by traversing the gray value of the pixel point included in each of the plurality of outer neighbors. And obtaining the maximum value of the plurality of outer neighborhoods by calculating the maximum value of the maximum gray values of each outer neighborhoods in the plurality of outer neighborhoods.

In operation S430, according to the maximum gray value, the feature value of the pixel point included in the target sub-region is calculated, and the feature value of the target sub-region is obtained.

According to the embodiment of the invention, the gray value of each pixel is obtained by traversing the pixel points included in the target subarea, and the characteristic value of each pixel is obtained according to the magnitude relation between the gray value of each pixel and the maximum gray value. And obtaining the characteristic value of the target sub-region by calculating the maximum value of the characteristic value of the pixel point included in the target sub-region.

In operation S440, the feature value of each target sub-region in the plurality of target sub-regions is combined to obtain the feature value set of the j-th first region, where the type of the target sub-region is an inner neighborhood.

According to the embodiment of the invention, the multiple target subareas of the jth first area are traversed, and when the target subareas are the outer neighborhoods, the characteristic values of the target subareas are empty sets; and when the target subarea is an inner neighborhood, combining the characteristic values of the target subarea according to a certain sequence to obtain a j-th first area characteristic value group. For example, by setting an initial preset feature value set, when traversing a plurality of target subregions of the jth first region, when the target subregion is an inner neighborhood, adding the feature value into the initial preset feature value set, updating the initial preset feature value set, and when traversing the plurality of target subregions, taking the updated preset feature value set as the jth first region feature value set.

In operation S450, the feature value sets of the plurality of first regions are combined to obtain the feature value set of the target region.

According to the embodiment of the invention, the positions of the first pixel points of the first areas are obtained by traversing the first areas, and the ordering result is obtained by ordering the positions of the first pixel points of the first areas. And combining the characteristic value sets of the plurality of first areas according to the sequencing result to obtain the characteristic value set of the target area.

According to the embodiment of the invention, for each first region, the characteristic value in each target subregion in the plurality of target subregions is obtained by calculating the maximum value of the gray value included in each target subregion in the plurality of target subregions in the first region, so that the characteristic value group corresponding to the plurality of first regions is obtained, and the calculation accuracy of the characteristic value group of the target region is improved through grading operation.

Fig. 5 schematically shows a flow chart for deriving a characteristic value of a target sub-region from a maximum gray value according to an embodiment of the invention.

In this embodiment, as shown in fig. 5, obtaining the characteristic value of the target sub-region according to the maximum gray value includes operations S551 to S552.

In operation S551, a difference value between each pixel point in the pixel points included in the target sub-region and the maximum gray value is calculated, so as to obtain a difference value group of the target sub-region.

According to the embodiment of the invention, the plurality of differences can be sequenced according to the position sequence of each pixel point in the pixel points included in the target subarea, so as to obtain a difference group.

In operation S552, the maximum value of the difference group is calculated, resulting in the feature value of the target sub-region.

According to the embodiment of the invention, when the difference value group is stored in the table, the difference value group can be calculated through the maximum value function in the table, so that the maximum value in the difference value group is obtained. When the difference value group is stored in the temporary array, the difference value group is calculated by calculating the function of the maximum value in the array, and the maximum value in the difference value group is obtained.

According to an embodiment of the present invention, for example, the maximum gray value is the maximum value of gray values of pixel points included in 3 outer neighborhoods, and the difference value calculation formula in the difference value group may be, for example, the following formula:

wherein,gray value representing pixel, +.>Representing +.>The characteristic value of the corresponding pixel point,represents maximizing the gray value of the pixel points included in the 3 outer neighbors, Representation->Difference from the maximum.

According to an embodiment of the present invention, the combining the feature values of each of the plurality of target subregions to obtain the feature value set of the j-th first region may include the following operations: sequencing each target subarea in the plurality of target subareas to obtain a first sequencing result; according to the first sorting result, sorting the characteristic value of each target subarea in the plurality of target subareas to obtain a second sorting result; and combining the characteristic values in the second sequencing result to obtain a characteristic value group of the j first region.

According to the embodiment of the invention, the first sorting result may be composed of positions of a plurality of pixel points and corresponding target sub-regions, and when the positions of the pixel points are represented by an abscissa and an ordinate together, the composition sequence may be a composition sequence obtained by first abscissa sorting and then ordinate sorting. For example, there are four target sub-regions, and the positions of the corresponding pixels are (100, 50), (200, 50), (100 ), (200, 100), respectively, and then the first ranking result may be ((100, 50), target sub-region 1), ((200, 50), target sub-region 2), ((100, 100), target sub-region 3), ((200, 100), target sub-region 4).

According to an embodiment of the present invention, the second sorting result may be composed of a plurality of feature values and corresponding target sub-regions.

According to the embodiment of the invention, the corresponding characteristic values of the target subareas in the first sorting result can be sorted according to the sorting results of the target subareas, so that a second sorting result is obtained.

According to an embodiment of the present invention, the feature value set may be composed of a plurality of feature values and corresponding pixel points. The feature values in the second ranking result may be combined in the order of the feature values in the second ranking result to obtain the feature value set of the j-th first region.

According to an embodiment of the present invention, combining the feature value sets of the plurality of first regions to obtain the feature value set of the target region may include the following operations: sequencing each first region in the plurality of first regions to obtain a third sequencing result; according to the third sorting result, sorting the characteristic value groups of each first region in the plurality of first regions to obtain a fourth sorting result; and combining the characteristic value groups in the fourth sorting result to obtain the characteristic value groups of the target area.

According to an embodiment of the present invention, the third ordering result may be composed of positions of a plurality of pixel points and corresponding first regions, and when the positions of the pixel points are indicated by the abscissa and the ordinate together, the composition sequence may be a composition sequence obtained by first abscissa ordering and then ordinate ordering.

According to an embodiment of the present invention, the fourth sorting result may be composed of a plurality of feature value sets and corresponding first regions.

According to the embodiment of the invention, the corresponding characteristic value groups can be ranked according to the ranking results of the plurality of first areas in the third ranking result, so as to obtain a fourth ranking result.

According to an embodiment of the present invention, the feature value set of the target area may be composed of a plurality of feature values and corresponding pixel points. And combining all the characteristic values in the characteristic groups according to the sequence of the characteristic values in the intermediate sorting result to obtain the characteristic value groups of the target region.

According to the embodiment of the invention, the characteristic value sets of the target area are obtained by sequencing each of the plurality of first areas and combining the characteristic value sets according to the sequencing result, so that the efficiency of detecting the abnormal points of the image is improved.

Fig. 6 schematically shows a flowchart of binarizing a target image to obtain a plurality of binary mask images of the target image according to an embodiment of the present invention.

In this embodiment, as shown in fig. 6, binarizing the target image to obtain a plurality of binary mask images of the target image includes operations S610 to S630.

In operation S610, for the kth first region, a gray value mean and a gray value standard deviation of pixel points included in the kth first region are calculated.

According to the embodiment of the invention, the total gray value is obtained by calculating the sum of the gray values of the pixel points included in the kth first region. And dividing the total gray value by the number of the pixel points included in the kth first area to obtain the gray value average value of the pixel points included in the kth first area.

According to an embodiment of the invention, the gray value standard deviation is the square root of the arithmetic mean from the gray value mean. When the brightness of the kth first area is larger and the contrast is larger, the gray value standard deviation is larger.

In operation S620, a plurality of gray threshold values of the kth first region are calculated according to the gray value mean value and the gray value standard deviation, and a gray threshold value group of the kth first region is obtained.

According to an embodiment of the present invention, the gray threshold calculation formula may be, for example, the following formula:

wherein,is gray value mean>Is gray value standard deviation +>Is a coefficient of->The value range of (2) is [3,5 ]]，/>Is the gray threshold.

According to the embodiment of the invention, the gray threshold value group of the kth first region is obtained by combining a plurality of gray threshold values according to the magnitude of the values.

In operation S630, binarization processing is performed on the target image using the gray threshold groups of the plurality of first regions, respectively, to obtain a plurality of binary mask images of the target image.

According to the embodiment of the invention, the total gray threshold group is obtained by combining the gray threshold in each gray threshold group in the plurality of gray threshold groups according to the value. And respectively carrying out binarization processing on the target image according to each gray threshold value in the total gray threshold value group to obtain a plurality of binary mask images of the target image, wherein a pixel point with a gray value of 1 in the binary mask images represents a first abnormal pixel point.

According to the embodiment of the invention, the target image is binarized by using the gray threshold value groups of the plurality of first areas, so that the binary mask image with different abnormal pixels can be obtained.

According to an embodiment of the present invention, calculating a first abnormal pixel group and a second abnormal pixel group of a feature value group of each of a plurality of binary mask images to obtain a target abnormal pixel in a target image may include the following operations: respectively carrying out intersection operation on a plurality of first abnormal pixel point groups of a plurality of binary mask images and a second abnormal pixel point group corresponding to the characteristic value group to obtain a plurality of intersection sets; and performing union operation on the intersection results to obtain a plurality of target abnormal pixel points in the target image.

According to the embodiment of the invention, according to the position of the first abnormal pixel point included in the first abnormal pixel point group, the position of the second abnormal pixel point included in the second abnormal pixel point group, when the first abnormal pixel point and the second abnormal pixel point have the same position, the pixel point is taken as the target abnormal pixel point. The method comprises the steps of performing intersection operation on a plurality of first abnormal pixel point groups and a second abnormal pixel point group of a characteristic value group respectively, and performing union operation on target abnormal pixel points in the intersection operation to obtain a plurality of target abnormal pixel points in a target image.

According to an embodiment of the present invention, the number of target areas may be plural. According to the first abnormal pixel point groups and the second abnormal pixel point groups obtained in each target area in the target areas, a plurality of abnormal pixel points corresponding to each target area can be obtained. And combining a plurality of abnormal pixel points of each target area in the plurality of target areas to obtain a plurality of target abnormal pixel points in the target image.

According to the embodiment of the invention, a plurality of intersection sets are obtained by respectively carrying out intersection operation on a plurality of first abnormal pixel point groups of a plurality of binary mask images and a second abnormal pixel point group of a characteristic value group, and a plurality of intersection results are subjected to union operation to obtain pixel points which are higher than an average gray value and have larger contrast with surrounding neighborhood as target abnormal pixel points, so that accurate abnormal pixel point positioning in a target image is obtained.

According to an embodiment of the present invention, the size of the target area may be 8 pixel points8 pixels by setting the scale of the sliding window to 8 pixels +.>8 pixel points, the step length is set to 8 pixel points, and 1 pixel point with the size of 8 pixel points is obtained by moving the position of the sliding window in the target area>A first region of 8 pixels.

According to an embodiment of the present invention, the first scale may be set to 4 pixel points4 pixel points can be traversed by traversing the first area from left to right and from top to bottom, in the traversing process, the traversing scale is the first scale, the first area is traversed by traversing the first area from left to right, the step length can be the width of the first scale, the first area is traversed by traversing the first area from top to bottom, the step length can be the length of the first scale, 4 first sub-areas are obtained by taking each traversed area as the first sub-area, and the size of the first sub-area is 4 pixel points->4 pixels.

According to an embodiment of the present invention, the second scale may be set to 2 pixel pointsThe 2 pixel points can traverse the 1 st first sub-area in sequence from left to right and from top to bottom, in the traversing process, the traversing scale is the second scale, the first sub-area is traversed in sequence from left to right, the step length is the width of the second scale, the first sub-area is traversed in sequence from top to bottom, the step length is the length of the second scale, and 4 target sub-areas are obtained by taking each traversed area as a target sub-area. And 1 st The way that one sub-region obtains the target sub-region is the same, the other 3 first sub-regions can be traversed to obtain the target sub-region included in each first sub-region in the other 3 first sub-regions, and the size of the target sub-region is 2 pixel points->2 pixels.

Fig. 7 schematically illustrates a schematic diagram of a multi-scale division result of a first region according to a specific embodiment of the present invention.

As shown in fig. 7, the multi-scale division result of the first region 710 may include a 1 st first sub-region 720, a 2 nd first sub-region 730, a 3 rd first sub-region 740, and a 4 th first sub-region 750, and the 1 st first sub-region 720 may include a 1 st target sub-region 720_1, a 2 nd target sub-region 720_2, a 3 rd target sub-region 720_3, and a 4 th target sub-region 720_4.

According to an embodiment of the present invention, the size of the first region 710 is 8 pixel pointsThe first region 710 has vertices of O11, O22, O32, and O42, and has a length and a width of 8 pixels.

According to an embodiment of the present invention, the 1 st first sub-region 720 corresponds to W1, W1 has a size of 4 pixel pointsThe 1 st first sub-region 720 is a region with 4 pixels in length and width, and takes O11, O12, O13 and I13 as vertexes.

According to an embodiment of the present invention, the 2 nd first sub-region 730 corresponds to W2, W2 has a size of 4 pixel pointsThe 2 nd first sub-region 730 is a region with 4 pixels in length and width, and takes O21, O22, O23 and I24 as vertices.

According to an embodiment of the invention, the 3 rd first sub-region 740 corresponds to W3, W3 has a size of 4 pixel pointsThe 3 rd first sub-region 740 is a region with the vertices of O31, O32, O33, and I32, and the length and width of the region are 4 pixels.

According to an embodiment of the present invention, the 4 th first sub-region 750 corresponds to W4, W4 has a size of 4 pixelsThe 4 th first sub-region 750 is a region with O41, O42, O43, and I41 as vertices and 4 pixels in length and width.

According to an embodiment of the present invention, the 1 st first sub-region 720, the 2 nd first sub-region 730, the 3 rd first sub-region 740, and the 4 th first sub-region 750 may each include 4 target sub-regions. The sizes of the 4 target subareas are all 2 pixel pointsThe 2 pixels, 4 target subregions may include 1 target subregion of type inner neighborhood and 3 target subregions of type outer neighborhood. The pixel points in the target sub-area with the type of the inner neighborhood can be represented by Isn, the value of Isn can be a gray value, s represents the serial number of the first sub-area, and n represents the serial number of the pixel points. The pixel points in the target subarea with the type of the outer neighborhood can be represented by Ost, the value of Ost can be a gray value, and t represents the sequence number of the target subarea. s, n and t are positive integers.

According to an embodiment of the present invention, the 1 st first sub-region 720 may include a 1 st target sub-region 720_1, a 2 nd target sub-region 720_2, a 3 rd target sub-region 720_3, and a 4 th target sub-region 720_4. The 1 st target sub-region 720_1 is a region of four pixels indicated by O11, the 2 nd target sub-region 720_2 is a region of four pixels indicated by O12, the 3 rd target sub-region 720_3 is a region of four pixels indicated by O13, and the 4 th target sub-region 720_4 is a region of four pixels indicated by I11, I12, I13, and I14.

Fig. 8 schematically shows an example schematic of image outlier detection according to an embodiment of the present invention.

As shown in fig. 8, an exemplary schematic diagram of image outlier detection includes operations S801-S817.

In operation S801, image data is acquired.

In operation S802, first preprocessing is performed on image data. The first pre-processing may be gray scale correction. For example, histogram statistics is performed on the gradation distribution of the image data, and the occurrence frequency of the gradation is counted in accordance with the magnitude of the value of the gradation. When the sum of the occurrence frequencies of the gray levels from small to large is higher than 10% of the total pixel number of the image data, the minimum gray level is obtained . When the sum of gray pixel frequencies from large to small is higher than 10% of the total pixel number of the image data, the maximum gray level +.>. And stretching correction is carried out on the region with concentrated gray distribution through double-threshold stretching, so that the influence of pixel points with gray values higher or lower than the average gray value on the stretching effect is avoided, and the contrast ratio of the image data is improved. The calculation formula of the dual-threshold stretching may be, for example, the following formula:

wherein,for the image data after the dual threshold stretching, +.>Is the acquired image data.

In operation S803, a second preprocessing is performed on the image data. The second preprocessing may be a blocking process, a rotation process, a flipping process. For example, the image data is divided into 512512-size area, when the area size is less than 512 +.>512, the size of the region is brought to 512 +.>512. The processing efficiency of S805 is improved by the blocking processing. The image data can be expanded through rotation processing and overturning processing, so that the problem of less acquired image data is solved.

In operation S804, an initial object detection model is built, and parameters that need to be updated are determined.

In operation S805, an initial model prediction result is acquired.

In operation S806, the model parameters are updated.

In operation S807, it is determined whether a convergence condition is reached? If the convergence condition is reached, the process advances to S808, and if not, the process advances to S805.

In operation S808, the updated parameter target detection model is verified.

In operation S809, it is determined whether the model verification is passed? In the case of passing the verification, S810 is entered, otherwise S801 is entered.

In operation S810, a target detection model is obtained.

In operation S811, a target image to be detected is acquired.

In operation S812, the object detection model is stored for use, and detection is performed using the object detection model.

In operation S813, a detection result is output.

In operation S814, a plurality of binary mask images of the target image are obtained according to the detection result.

In operation S815, a feature value set of the target area is obtained according to the detection result.

In operation S816, a plurality of binary mask images and feature value sets are calculated.

In operation S817, a target abnormal pixel point in the target image is obtained.

Based on the image abnormal point detection method, the invention also provides an image abnormal point detection device. The device will be described in detail below in connection with fig. 9.

Fig. 9 schematically shows a block diagram of the structure of an image outlier detection apparatus according to an embodiment of the present invention.

As shown in fig. 9, the image outlier detection apparatus 900 of the embodiment includes a detection module 910, a sliding window processing module 920, and a binarization processing module 930, a multi-scale division module 940, a first calculation module 950, and a second calculation module 960.

The detection module 910 is configured to detect an obtained target image by using a target detection model to obtain a target area in the target image, where the target detection model is obtained by training an initial target detection model according to a training sample, the training sample includes a plurality of images related to a sample image and a plurality of labels, and the labels represent types of the sample area in the image.

The sliding window processing module 920 is configured to perform sliding window processing on the target area to obtain a plurality of first areas.

The binarization processing module 930 is configured to perform binarization processing on the target image according to the gray values of the pixel points included in each first area, so as to obtain a plurality of binary mask images of the target image, where a pixel point with a gray value of 1 in the binary mask image is a first abnormal pixel point.

The multi-scale division module 940 is configured to perform multi-scale division on the ith first region to obtain a plurality of target subregions of the ith first region, where the types of the target subregions include an inner neighborhood or an outer neighborhood, gray values of each pixel point in the inner neighborhood are different, gray values of each pixel point in the outer neighborhood are identical, and i is a positive integer.

The first calculating module 950 is configured to calculate, according to gray values of pixels included in a plurality of target sub-regions of the plurality of first regions, feature values corresponding to each of the target sub-regions, respectively, to obtain a feature value set of the target region, where a pixel corresponding to the feature value in the feature value set is a second abnormal pixel.

The second calculation module 960 is configured to calculate a first abnormal pixel group of each of the plurality of binary mask images and a second abnormal pixel group of the feature value group, so as to obtain a target abnormal pixel in the target image.

According to an embodiment of the present invention, the multi-scale division module 940 includes: a first dividing unit and a second dividing unit.

The first dividing unit is used for dividing the ith first area on a first scale to obtain a plurality of first sub-areas.

The second dividing unit is used for dividing each first subarea in the plurality of first subareas on a second scale to obtain a plurality of target subareas of the ith first area, wherein the second scale is smaller than the first scale.

According to an embodiment of the invention, the first calculation module 950 includes: the device comprises a first determining unit, a first calculating unit, a second calculating unit, a first combining unit and a second combining unit.

The first determining unit is used for determining a plurality of outer neighborhoods of the target subarea according to the j-th first area under the condition that the target subarea is determined to be the inner neighborhoods, wherein j is a positive integer.

The first calculating unit is used for calculating the maximum value of the gray values of the pixel points included in the plurality of outer neighborhoods and obtaining the maximum gray values of the plurality of outer neighborhoods.

And the second calculation unit is used for calculating the characteristic value of the pixel point included in the target subarea according to the maximum gray value to obtain the characteristic value of the target subarea.

The first combination unit is used for combining the characteristic values of each target subarea in the plurality of target subareas to obtain a characteristic value group of the j first area, wherein the type of the target subarea is an inner neighborhood.

And the second combination unit is used for combining the characteristic value groups of the plurality of first areas to obtain the characteristic value group of the target area.

According to an embodiment of the present invention, the second calculation unit includes: a first computing subunit and a second computing subunit.

The first calculating subunit is configured to calculate a difference value between each pixel point in the pixel points included in the target sub-area and the maximum gray value, and obtain a difference value group of the target sub-area.

And the second calculating subunit is used for calculating the maximum value of the difference value group to obtain the characteristic value of the target subarea.

According to an embodiment of the present invention, the first combining unit includes: a first ordering subunit, a second ordering subunit, and a first combining subunit.

And the first sequencing subunit is used for sequencing each target subarea in the plurality of target subareas to obtain a first sequencing result.

And the second sorting subunit is used for sorting the characteristic value of each target subarea in the plurality of target subareas according to the first sorting result to obtain a second sorting result.

And the first combination subunit is used for combining the characteristic values in the second sequencing result to obtain a characteristic value group of the j first region.

According to an embodiment of the present invention, the second combining unit includes: a third sorting subunit, a fourth sorting subunit, and a second combining subunit.

And the third sequencing subunit is used for sequencing each first region in the plurality of first regions to obtain a third sequencing result.

And the fourth sorting subunit is used for sorting the characteristic value groups of each first area in the plurality of first areas according to the third sorting result to obtain a fourth sorting result.

And the second combination subunit is used for combining the characteristic value groups in the fourth sorting result to obtain the characteristic value groups of the target area.

According to an embodiment of the present invention, the binarization processing module 930 includes: a third calculation unit, a fourth calculation unit and a binarization processing unit.

And a third calculation unit, configured to calculate, for the kth first area, a gray value average value and a gray value standard deviation of pixel points included in the kth first area, where k is a positive integer.

And the fourth calculation unit is used for calculating a plurality of gray threshold values of the kth first area according to the gray value mean value and the gray value standard deviation to obtain a gray threshold value group of the kth first area.

And the binarization processing unit is used for respectively carrying out binarization processing on the target image by utilizing the gray threshold value groups of the plurality of first areas to obtain a plurality of binary mask images of the target image.

According to an embodiment of the invention, the second calculation module 960 includes: an intersection operation unit and a union operation unit.

And the intersection operation unit is used for respectively carrying out intersection operation on the first abnormal pixel point groups of the binary mask images and the second abnormal pixel point groups of the characteristic value groups to obtain a plurality of intersection sets.

And the union operation unit is used for performing union operation on the plurality of intersection sets to obtain a plurality of target abnormal pixel points in the target image.

Any of the detection module 910, the sliding window processing module 920, and the binarization processing module 930, the multi-scale division module 940, the first calculation module 950, and the second calculation module 960 may be combined in one module to be implemented, or any of the modules may be split into a plurality of modules according to an embodiment of the present invention. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module. According to embodiments of the invention, at least one of the detection module 910, the sliding window processing module 920, and the binarization processing module 930, the multi-scale division module 940, the first calculation module 950, and the second calculation module 960 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or in hardware or firmware in any other reasonable manner of integrating or packaging the circuitry, or in any one of or a suitable combination of three of software, hardware, and firmware. Alternatively, at least one of the detection module 910, the sliding window processing module 920, and the binarization processing module 930, the multi-scale division module 940, the first calculation module 950, and the second calculation module 960 may be at least partially implemented as computer program modules, which when executed, may perform the corresponding functions.

As shown in fig. 10, the electronic device according to the embodiment of the present invention includes a processor 1001 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. The processor 1001 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 1001 may also include on-board memory for caching purposes. The processor 1001 may include a single processing unit or a plurality of processing units for performing different actions of the method flow according to an embodiment of the invention.

In the RAM 1003, various programs and data necessary for the operation of the electronic apparatus are stored. The processor 1001, the ROM1002, and the RAM 1003 are connected to each other by a bus 1004. The processor 1001 performs various operations of the method flow according to the embodiment of the present invention by executing programs in the ROM1002 and/or the RAM 1003. Note that the program may be stored in one or more memories other than the ROM1002 and the RAM 1003. The processor 1001 may also perform various operations of the method flow according to an embodiment of the present invention by executing programs stored in the one or more memories.

According to an embodiment of the invention, the electronic device may further comprise an input/output (I/O) interface 1005, the input/output (I/O) interface 1005 also being connected to the bus 1004. The electronic device may also include one or more of the following components connected to an input/output (I/O) interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output portion 1007 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk or the like; and a communication section 1009 including a network interface card such as a LAN card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The drive 1010 is also connected to an input/output (I/O) interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is installed as needed in the drive 1010, so that a computer program read out therefrom is installed as needed in the storage section 1008.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that the features recited in the various embodiments of the invention and/or in the claims may be combined in various combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the invention. In particular, the features recited in the various embodiments of the invention and/or in the claims can be combined in various combinations and/or combinations without departing from the spirit and teachings of the invention. All such combinations and/or combinations fall within the scope of the invention.

The embodiments of the present invention are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the invention is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the invention, and such alternatives and modifications are intended to fall within the scope of the invention.

Claims

1. An image outlier detection method, comprising:

detecting an acquired target image by using a target detection model to obtain a target region in the target image, wherein the target detection model is obtained by training an initial target detection model according to a training sample, the training sample comprises a plurality of images related to a sample image and a plurality of labels, and the labels represent the types of the sample region in the image;

Carrying out sliding window treatment on the target area to obtain a plurality of first areas;

respectively carrying out binarization processing on the target image according to the gray value of the pixel point included in each first area to obtain a plurality of binary mask images of the target image, wherein the pixel point with the gray value of 1 in the binary mask images is a first abnormal pixel point;

for an ith first area, carrying out multi-scale division on the ith first area to obtain a plurality of target subareas of the ith first area, wherein the types of the target subareas comprise an inner neighborhood or an outer neighborhood, the gray values of each pixel point in the inner neighborhood are different, the gray values of each pixel point in the outer neighborhood are the same, and i is a positive integer;

calculating characteristic values corresponding to each target sub-region according to gray values of pixel points included in the target sub-regions of the first regions respectively to obtain a characteristic value group of the target region, wherein the pixel points corresponding to the characteristic values in the characteristic value group are second abnormal pixel points;

the calculating, according to gray values of pixel points included in the target sub-regions of the first regions, feature values corresponding to the target sub-regions respectively, to obtain a feature value set of the target region includes:

For a j-th first region, determining a plurality of outer neighborhoods of the target sub-region under the condition that the target sub-region is determined to be the inner neighborhoods, wherein j is a positive integer;

calculating the maximum value of gray values of the pixel points included in the plurality of outer neighborhoods to obtain the maximum gray values of the plurality of outer neighborhoods;

calculating the characteristic value of the pixel point included in the target subarea according to the maximum gray value to obtain the characteristic value of the target subarea;

combining the characteristic values of each target subarea in the target subareas to obtain a characteristic value group of the j-th first area, wherein the type of the target subarea is the inner neighborhood;

combining the characteristic value sets of the plurality of first areas to obtain the characteristic value set of the target area;

and calculating the first abnormal pixel point group and the second abnormal pixel point group of the characteristic value group of each of the plurality of binary mask images to obtain target abnormal pixel points in the target image.

2. The method according to claim 1, wherein the multi-scale dividing the ith first region for the ith first region to obtain a plurality of target sub-regions of the ith first region includes:

Dividing the ith first region on a first scale to obtain a plurality of first sub-regions;

dividing each first subarea in the first subareas on a second scale to obtain a plurality of target subareas of the ith first area, wherein the second scale is smaller than the first scale.

3. The method according to claim 1, wherein the calculating the characteristic value of the pixel point included in the target sub-region according to the maximum gray value to obtain the characteristic value of the target sub-region includes:

calculating the difference value between each pixel point in the pixel points included in the target subarea and the maximum gray value to obtain a difference value group of the target subarea;

and calculating the maximum value of the difference value group to obtain the characteristic value of the target subarea.

4. The method of claim 1, wherein combining the feature values of each of the plurality of target subregions to obtain the set of feature values for the j-th first region comprises:

sequencing each target subarea in the target subareas to obtain a first sequencing result;

according to the first sorting result, sorting the characteristic value of each target subarea in the plurality of target subareas to obtain a second sorting result;

And combining the characteristic values in the second sequencing result to obtain the characteristic value group of the j-th first region.

5. The method of claim 1, wherein combining the feature value sets of the plurality of first regions to obtain the feature value set of the target region comprises:

sequencing each first region in the plurality of first regions to obtain a third sequencing result;

according to the third sorting result, sorting the characteristic value groups of each first region in the plurality of first regions to obtain a fourth sorting result;

and combining the characteristic value groups in the fourth sorting result to obtain the characteristic value groups of the target area.

6. The method according to claim 1, wherein the binarizing the target image according to the gray level of the pixel point included in each first region to obtain a plurality of binary mask images of the target image includes:

for a kth first region, calculating a gray value mean value and a gray value standard deviation of pixel points included in the kth first region, wherein k is a positive integer;

calculating a plurality of gray threshold values of the kth first region according to the gray value mean value and the gray value standard deviation to obtain a gray threshold value group of the kth first region;

And respectively carrying out binarization processing on the target image by using the gray threshold value groups of the first areas to obtain a plurality of binary mask images of the target image.

7. The method according to claim 1, wherein the calculating the first abnormal pixel group of each of the plurality of binary mask images and the second abnormal pixel group of the feature value group to obtain the target abnormal pixel in the target image includes:

respectively carrying out intersection operation on a plurality of first abnormal pixel point groups of the binary mask images and a second abnormal pixel point group corresponding to the characteristic value group to obtain a plurality of intersection sets;

and performing union operation on the intersection sets to obtain a plurality of target abnormal pixel points in the target image.

8. An image outlier detection apparatus, comprising:

the detection module is used for detecting an acquired target image by using a target detection model to obtain a target area in the target image, wherein the target detection model is obtained by training an initial target detection model according to a training sample, the training sample comprises a plurality of images related to a sample image and a plurality of labels, and the labels represent the types of the sample areas in the image;

The sliding window processing module is used for carrying out sliding window processing on the target area to obtain a plurality of first areas;

the binarization processing module is used for performing binarization processing on the target image according to the gray value of the pixel point included in each first area to obtain a plurality of binary mask images of the target image, wherein the pixel point with the gray value of 1 in the binary mask images is a first abnormal pixel point; the multi-scale division module is used for carrying out multi-scale division on the ith first area aiming at the ith first area to obtain a plurality of target subareas of the ith first area, wherein the types of the target subareas comprise inner neighborhoods or outer neighborhoods, the gray values of each pixel point in the inner neighborhoods are different, the gray values of each pixel point in the outer neighborhoods are the same, and i is a positive integer;

the first calculation module is used for calculating the characteristic value corresponding to each target sub-region according to the gray values of the pixel points included in the target sub-regions of the first regions respectively to obtain a characteristic value group of the target region, wherein the pixel points corresponding to the characteristic values in the characteristic value group are second abnormal pixel points;

and the second calculation module is used for calculating the first abnormal pixel point groups of the binary mask images and the second abnormal pixel point groups of the characteristic value groups to obtain target abnormal pixel points in the target image.

9. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-7.