WO2022012573A1

WO2022012573A1 - Image processing method and apparatus, electronic device, and storage medium

Info

Publication number: WO2022012573A1
Application number: PCT/CN2021/106200
Authority: WO
Inventors: 马欣; 祝夭龙
Original assignee: 北京灵汐科技有限公司
Priority date: 2020-07-14
Filing date: 2021-07-14
Publication date: 2022-01-20
Also published as: CN113938671B; CN113938671A

Abstract

The present invention provides an image processing method, comprising: performing area division on a first image to obtain a plurality of image subareas, wherein an overlapping area exists between adjacent image subareas in the plurality of image subareas; determining subarea quality scores of the image subareas, the subarea quality scores representing the image quality of the image subareas; and according to the subarea quality scores of the image subareas, determining an image quality score of the first image, the image quality score representing the image quality of the first image. The present invention further provides an image processing method, an image processing apparatus, an electronic device, and a readable storage medium.

Description

Image processing method and device, electronic device, storage medium

technical field

The present disclosure relates to the technical field of image processing, and in particular, to an image processing method, an image processing apparatus, an electronic device, and a readable storage medium.

Background technique

Image content analysis is a common image processing technique. The main process of image content analysis is to obtain video images from the camera, decode the video images, and then perform content analysis on each decoded image.

In some related technologies, when performing image content analysis, it is necessary to evaluate the image quality first, and then perform content analysis for images with higher image evaluation quality. If the image quality evaluation is inaccurate, the accuracy of the content analysis of the image will be reduced, and the efficiency of the content analysis of the image may also be reduced. It can be seen that the accuracy of image quality evaluation is a key factor for the content analysis of images.

How to improve the accuracy of image quality evaluation has become the pursuit of this field.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure provide an image processing method, an image processing apparatus, an electronic device, and a readable storage medium.

In a first aspect, an embodiment of the present disclosure provides an image processing method, including:

Dividing the first image into regions to obtain a plurality of image sub-regions, wherein adjacent image sub-regions in the plurality of image sub-regions have overlapping regions;

determining a sub-region quality score of each of the image sub-regions, the sub-region quality score representing the image quality of the image sub-region;

The image quality score of the first image is determined according to the sub-region quality score of each of the image sub-regions, and the image quality score represents the image quality of the first image.

In a second aspect, an embodiment of the present disclosure provides an image processing method, including:

Perform content analysis on the image to be analyzed to obtain content analysis results;

When the content analysis result indicates that the image to be analyzed includes a target object, target position information is generated, wherein the target position information represents the position of the target object in the image to be analyzed, and the target The location information is used to determine the location weights of multiple image sub-regions in the first image when determining the image quality score of the first image, where the first image is an image collected after the image to be analyzed, and multiple The adjacent image sub-regions in the image sub-regions have overlapping regions.

In a third aspect, embodiments of the present disclosure provide an image processing apparatus, including:

a dividing module, configured to divide the first image into regions to obtain a plurality of image sub-regions, wherein adjacent image sub-regions in the plurality of image sub-regions have overlapping regions;

a calculation module, configured to determine a sub-region quality score of each of the image sub-regions, where the sub-region quality score represents the image quality of the image sub-region;

An acquisition module, configured to determine an image quality score of the first image according to the sub-region quality scores of each of the image sub-regions, where the image quality score represents the image quality of the first image.

In a fourth aspect, embodiments of the present disclosure provide an image processing apparatus, including:

The analysis module is used to analyze the content of the image to be analyzed, and obtain the content analysis result;

The feedback module is configured to generate target position information when the content analysis result indicates that the image to be analyzed includes a target object, wherein the target position information represents the position of the target object in the image to be analyzed. position, the target position information is used to determine the position weight of a plurality of image sub-regions in the first image when the image quality score of the first image is determined, and the first image is collected after the image to be analyzed image, and adjacent image sub-regions among the plurality of image sub-regions have overlapping regions.

In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including: a memory, a processor, and a program or instruction stored on the memory and executable on the processor, where the program or instruction is processed by the processor implements at least one of the following methods when executing:

Any image processing method provided by the first aspect of the embodiments of the present disclosure;

Any image processing method provided by the second aspect of the embodiments of the present disclosure.

In a sixth aspect, an embodiment of the present disclosure provides a readable storage medium, where a program or an instruction is stored thereon, and when the program or instruction is executed by a processor, at least one of the following methods is implemented:

In the embodiment of the present disclosure, since the sub-region quality scores of each image sub-region are determined respectively, the image quality of each image sub-region can be evaluated individually, and then the image of the first image is obtained according to the sub-region quality scores of each image sub-region The quality score, wherein when the image sub-regions are divided, adjacent image sub-regions have overlapping areas, which can ensure the continuity of the evaluation image content of each image sub-region, so that the image quality score of the first image can more accurately reflect the first image. The image quality of an image.

When the image quality score obtained in the image processing method provided by the first aspect of the present disclosure is applied to image screening based on image quality, the accuracy of image screening can be improved, for example, images with unqualified image quality can be accurately screened out , only perform content analysis on images with qualified image quality, thereby reducing the power consumption of image content analysis, improving the efficiency of image content analysis, and improving the accuracy of image content analysis.

Description of drawings

1 is a flowchart of an image processing method provided by an embodiment of the present disclosure;

2 is a flowchart of some steps in another image processing method provided by an embodiment of the present disclosure;

3 is a flowchart of some steps in another image processing method provided by an embodiment of the present disclosure;

4 is a flowchart of some steps in still another image processing method provided by an embodiment of the present disclosure;

5 is a flowchart of some steps in still another image processing method provided by an embodiment of the present disclosure;

6 is a flowchart of some steps in still another image processing method provided by an embodiment of the present disclosure;

7 is a flowchart of some steps in still another image processing method provided by an embodiment of the present disclosure;

8 is a flowchart of some steps in still another image processing method provided by an embodiment of the present disclosure;

9 is a flowchart of an image processing method provided by an embodiment of the present disclosure;

10 is a schematic diagram of an image processing in an embodiment of the present disclosure;

11 is a schematic diagram of an image area division in an embodiment of the present disclosure;

12 is a flowchart of an image processing method provided by an embodiment of the present disclosure;

13 is a structural diagram of an image processing apparatus provided by an embodiment of the present disclosure;

14 is a structural diagram of an image processing apparatus provided by an embodiment of the present disclosure;

FIG. 15 is a structural diagram of an electronic device provided by an embodiment of the present disclosure.

detailed description

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The terms "first", "second" and the like in the description and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in sequences other than those illustrated or described herein, and "first", "second" distinguishes Usually it is a class, and the number of objects is not limited. For example, the first object may be one or multiple.

Referring to FIG. 1, FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure, as shown in FIG. 1, including the following steps:

S100. Divide the first image into regions to obtain multiple image sub-regions, wherein adjacent image sub-regions among the multiple image sub-regions have overlapping regions.

Wherein, the above-mentioned first image may be any frame of image obtained by decoding an image collected by a camera, and further, the above-mentioned first image may be a current image, for example, an image frame decoded and output when step 101 is executed. In addition, the above-mentioned first image may be an image collected by a built-in or external camera of the electronic device. This embodiment of the present disclosure does not limit this. For example, the image may also be received through a network, and the above-mentioned decoding may be performed locally or remotely.

The above-mentioned division of regions may be performed according to preset division positions to obtain s regions, where s may be a preset integer greater than 1, for example: 2, 4, 6... Specifically, it can be set according to the application scenario. Certainly.

In the above-mentioned s image sub-regions, adjacent image sub-regions have overlapping regions.

The overlapping area exists in the above-mentioned adjacent image sub-areas may be, the adjacent area of the adjacent image sub-areas is an overlapping area.

For example, taking the above s as 4 as an example, assuming that the first image from the decoded video is I, perform sub-region segmentation of m rows and n columns for I. After the division, there are s=m×n image sub-regions in total, and each image sub-region overlaps with a width of t pixels. The parameters m, n and t are all preset values according to actual service requirements, and the overlapping situation of the divided image sub-regions is shown in Figure 11.

Since there are overlapping areas in adjacent image sub-areas, when determining the sub-area quality score of each image sub-area, it can ensure the continuity of the evaluated image content of each image sub-area, and avoid the same object being divided into different image sub-areas. deviation, so that the image quality score determined according to the sub-region quality score of each image sub-region can more accurately reflect the image quality.

S200. Determine the sub-region quality score of each image sub-region, where the sub-region quality score represents the image quality of the image sub-region.

In this step, the sub-area quality score may be calculated separately for each image sub-area, and specifically may be calculated based on the image content in each image sub-area.

S300. Determine an image quality score of the first image according to the sub-region quality scores of each image sub-region, where the image quality score represents the image quality of the first image.

In this step, the sub-region quality scores of each image sub-region may be summed to obtain the total score of the first image; or the sub-region quality scores of each image sub-region may be summed and averaged to obtain the first image sub-region quality score. The total score of an image; or the quality score of each sub-region of the image can be multiplied by the corresponding weight, and then summed to obtain the total score of the first image. This embodiment of the present disclosure does not limit this.

Since the sub-region quality scores of each image sub-region are determined separately, the image quality of each image sub-region can be evaluated individually, and then the image quality score of the first image is obtained according to the sub-region quality scores of each image sub-region, wherein, in When the image sub-regions are divided, adjacent image sub-regions have overlapping areas, which can ensure the continuity of the evaluated image content of each image sub-region, so that the image quality score of the first image can more accurately reflect the image quality of the first image.

In the embodiment of the present disclosure, the image quality score determined through the above steps can be applied to image screening based on image quality, and can improve the accuracy of image screening. It can reduce the power consumption of image content analysis, improve the efficiency of image content analysis, and improve the accuracy of image content analysis.

It should be noted that the embodiments of the present disclosure may be applied to electronic devices, such as embedded devices, mobile phones, tablet computers, wearable devices, vehicles, etc., which are not limited in the embodiments of the present disclosure.

As an optional implementation manner, referring to FIG. 2 , determining the image quality score of the first image according to the sub-region quality scores of each image sub-region includes the following steps:

S310, weighting the sub-region quality scores of each image sub-region according to the position weight of each image sub-region, and summing them to obtain the image quality score of the first image, wherein the position weight of the image sub-region indicates that the image sub-region is in the first image sub-region The importance of a location in an image.

The position weight of each image sub-region may be preset according to the position of the image sub-region in the first image. For example, in an actual monitoring or shooting scene, the upper part of the image often corresponds to the sky or the top of the room, while the lower part of the image often corresponds to the sky or the top of the room. Corresponding to effective objects such as people or vehicles, the position weight of the upper part of the image can be set to be smaller than the position weight of the lower part of the image. This is just a simple example. The position weight of each image sub-region can be specifically set according to the requirements of the application scenario.

The above-mentioned determination of the image quality score of the first image according to the position weight of each image sub-region and the sub-region quality score may be, weighting the sub-region quality scores of each image sub-region according to the position weight of each image sub-region, and summing, to obtain the image quality score of the first image. In the embodiment of the present disclosure, the larger the position weight is, the more important the position of the image sub-region in the first image is.

In the embodiment of the present disclosure, the image quality score of the first image is determined according to the position weight of each image sub-region and the sub-region quality score, which can implement a more refined evaluation of the image quality of the first image.

As an optional implementation manner, referring to FIG. 3 , before the step of determining the image quality score of the first image according to the position weights and subregion quality scores of each image subregion, determine the subregion quality score according to the subregion quality scores of each image subregion. The image quality score of the first image further includes the following steps:

S320. Obtain target location information, where the target location information represents the location of the target object in the second image determined by performing content analysis on the second image, and the second image is an image collected before the first image.

When image quality evaluation and image content analysis are performed in different devices/equipment, the target location information may be generated and sent by the device/device performing image content analysis, and the target location information may be acquired by the device/device performing image quality evaluation Receive target location information from an apparatus/equipment that performs image content analysis; when image quality evaluation and image content analysis are performed in the same apparatus/equipment, acquiring target location information may be acquiring target location information from a storage device; The location information is directly applied to the image quality evaluation process as a parameter. This embodiment of the present disclosure does not limit this.

The above-mentioned target object may be a preset monitored object, or an image object conforming to preset characteristics, such as a person, a vehicle, and the like.

The above-mentioned second image may be an image in the same video as the first image, and the second image is acquired before the first image. For example, the second image is an image of the previous frame of the first image, or the second image is an image multiple frames before the first image.

S330. Determine the position weight of each image sub-region according to the target position information.

Determining the position weight of each image sub-region according to the target position information may be that the position weight of the image sub-region in the first image corresponding to the position of the target object in the second image is set to be greater than that of other image sub-regions in the first image. position weight. This is just a simple example. The position weight of each image sub-region can be specifically set according to the requirements of the application scenario.

In the process of image quality evaluation, the position of the target object has a great influence on the image quality. On the basis of determining the position of the target object in the image frame before the first image through content analysis, each image is determined according to the target position information. The position weight of the sub-region and further determining the image quality score of the first image can improve the accuracy of the image quality evaluation.

As an optional implementation manner, referring to FIG. 4 , determining the position weight of each image sub-region according to the target position information includes the following steps:

S331. Divide the plurality of image sub-areas into at least one first target image sub-area and at least one second target image sub-area according to the target position information, where the first target image sub-area is the same as the target object in the first image and the second target image sub-area. The subregion of the image corresponding to the location in the image.

The second target image sub-region is an image sub-region other than the first target image sub-region in the first image.

S332. Determine the second weight of each first target image sub-region, where the second weight of the first target image sub-region represents the possibility that the first target image sub-region includes the target object.

Determining the second weight of each sub-region of the first target image may be to set the second weight of the sub-region of the first target image according to target position information generated by performing content analysis on the second image. Therefore, the second weight may also be referred to as a feedback weight.

S333. Determine the position weight of each first target image sub-region according to the first weight of each first target image sub-region and the second weight of each first target image sub-region.

Determining the position weight of the first target image sub-area according to the first weight and the second weight of the first target image sub-area may be that the position weight of the first target image sub-area is equal to the sum of the first weight and the second weight, or the The position weight of a target image sub-region is a value obtained by normalizing the sum of the first weight and the second weight.

S334: Determine the first weight of each second target image sub-region as the position weight of each second target image sub-region.

The first weight of the image sub-region represents the importance of the position of the image sub-region in the first image.

The first weight of each image sub-region may be preset according to the position of the image sub-region in the first image. For example, in an actual monitoring or shooting scene, the upper part of the image often corresponds to the sky or the top of the room, while the lower part of the image often corresponds to the sky or the top of the room. It corresponds to an effective object such as a person or a vehicle, so that the first weight of the upper part of the image can be set to be smaller than the first weight of the lower part of the image. This is just a simple example. The first weight of each image sub-region can be specifically set according to the requirements of the application scenario.

In this embodiment, the second weight is determined according to the target position information generated by the content analysis of the second image, so that the position weight of the sub-region of the first target image can more accurately characterize the sub-region of the first target image in the first image. the importance of the location in the .

As an optional implementation manner, referring to FIG. 5 , the position weight of each image sub-region is determined according to the target position information, further comprising the following steps:

S335 , normalize the position weights of each image sub-region, so that the sum of the position weights of each image sub-region is equal to 1.

As an optional implementation manner, referring to FIG. 6 , determining the second weight of each first target image sub-region includes the following steps:

S3321. Determine the second weight of each first target image sub-region according to the time interval between the first image and the second image, the second weight of the first target image sub-region and the time between the first image and the second image Intervals are negatively correlated.

The negative correlation between the second weight and the time interval between the first image and the second image may be that when the first image is closer to the second image (the interval between the first image and the second image is less or no image ), the larger the second weight is, and vice versa. The time interval here may be represented by the number of image frames in the interval between the first image and the second image. The negative correlation between the second weight and the time interval between the first image and the second image can be realized. When the first image is closer to the second image, the more likely the target object is in the first target sub-area, the more likely the target object is in the first target sub-region. The larger the second weight of a target sub-region is set, the more important the first target sub-region is.

In this embodiment, the second weight is negatively correlated with the time interval between the first image and the second image, so that the position weight of the sub-region of the first target image can more accurately characterize the sub-region of the first target image in the first image. the importance of the location in the .

As an optional implementation manner, referring to FIG. 7 , determining the sub-region quality score of each image sub-region includes the following steps:

S210: Distinguish the image quality of the sub-regions of the image according to at least one image quality determination algorithm, and obtain initial quality scores of the sub-regions corresponding to various image quality determination algorithms.

The image quality determination algorithm may be at least one of Laplacian algorithm and Sobel algorithm. This is just a simple example. The number and type of image quality discrimination algorithms can be set according to the requirements of the application scenario. For example, the image quality discrimination algorithm can be replaced according to the actual service, and the discrimination of more image discrimination indicators (such as brightness, color, etc.) can be added.

S220. Determine the sub-region quality score of the image sub-region according to the initial quality score of each sub-region.

Determining the sub-region quality scores of the image sub-regions according to the initial quality scores of the sub-regions may be: summing the initial quality scores of the sub-regions to obtain the sub-region quality scores of the image sub-regions; The scores are summed and averaged to obtain the sub-region quality scores of the image sub-regions; alternatively, the initial quality scores of each sub-region are weighted and then summed to obtain the sub-region quality scores of the image sub-regions. This embodiment of the present disclosure does not limit this.

In the embodiment of the present disclosure, the image quality of the image sub-regions is discriminated according to at least one image quality discriminating algorithm, which can facilitate the upgrading of the image quality discriminating algorithm, and can discriminate against various indicators through different image quality discriminating algorithms. , so that the sub-region quality score can more accurately characterize the image quality of the image sub-region.

As an optional implementation manner, referring to FIG. 8 , after the step of determining the image quality score of the first image according to the sub-region quality scores of each image sub-region, the method further includes the following steps:

S401. Determine whether the image quality score is less than a preset threshold.

The above-mentioned preset threshold may be an experience value set by a user according to experience, or may be a threshold value learned according to an analysis result of historical image content, etc., which is not limited in this embodiment of the present disclosure.

S402. In the case that the image quality score is less than a preset threshold, discard the first image.

It can be understood that the above-mentioned image quality score is less than the preset threshold value, that the quality of the first image does not meet the preset condition, or the quality of the first image is unqualified. The foregoing discarding the first image may be understood as not performing a content analysis operation on the foregoing first image, for example, deleting or skipping the first image. Through step S402, it is possible to avoid invalid calculation caused by content analysis on images with unqualified quality.

S403. In the case that the image quality score is greater than or equal to a preset threshold, use the first image as the image to be analyzed.

It can be understood that the above-mentioned image quality score is greater than or equal to the preset threshold value, it can be understood that the quality of the first image satisfies the preset condition, or the quality of the first image is qualified.

The above image to be analyzed can be understood as an image for content analysis. In the embodiment of the present disclosure, the content analysis operation is not limited, for example, the content analysis operation may be a currently existing content analysis operation or the content analysis operation may be a content analysis operation of subsequent evolution.

FIG. 9 is a flowchart of an image processing method provided by an embodiment of the present disclosure. As shown in FIG. 9 , the method includes the following steps:

S501. Perform content analysis on the image to be analyzed to obtain a content analysis result.

S502. In the case where the content analysis result indicates that the image to be analyzed includes the target object, generate target position information, wherein the target position information represents the position of the target object in the image to be analyzed, and the target position information is used to determine the position of the first image. The position weights of multiple image sub-regions in the first image are determined when the image quality is scored. The first image is an image collected after the image to be analyzed, and adjacent image sub-regions in the multiple image sub-regions have overlapping regions.

In this embodiment of the present disclosure, the image to be analyzed is an image with an image quality score greater than a preset threshold, and the image quality score is determined through steps S100 to S300. When the image quality score is greater than or equal to the preset threshold, it can be understood that the quality of the image to be analyzed satisfies the preset condition, or the quality of the image to be analyzed is qualified.

In the embodiment of the present disclosure, content analysis is only performed on the images that meet the preset conditions or the quality is qualified, which can avoid the invalid calculation caused by the content analysis of the unqualified images, thereby improving the content analysis efficiency; The target position information is generated when the target object is generated, which is beneficial to improve the accuracy of image screening in the process of image screening based on image quality.

One implementation is as follows:

According to the actual business scenario, the first image is divided into S image sub-regions, and the position weight of the image sub-regions is represented as α. Preset subregions each image I _s based on the importance of each sub-image regions I _s position on the first image first weight (referred to as spatial or weights) α _space, in the range α _space may be 0≤α _space ≤1. The position weight α of the first target image sub-region can be expressed as α=α _space +α _feedback . Among them, α _feedback represents the second weight (or feedback weight) of the sub-region of the first target image, which represents the influence of the position of the target object in the second image on the quality evaluation of the first image, and the value range of _{α feedback can be} is 0.001≤α _feedback ≤0.5. Among them, _{the acquisition method of α feedback} can be as follows:

Assuming that each consecutive m frame is a time statistical period, the position of the target object found in the second image is obtained from the content analysis result, and recorded as the key position; for the image sub-region in the first image corresponding to the above key position, that is For the first target image sub-region, calculate the second weight of the first target image sub-region in the current statistical period according to the exponential decay method:

α _feedback (t)=exp[-θ(t+1)]

Among them, α _feedback (t) indicates that with the change of time t, the α _{feedback of the} first target image sub-region decays with time, and the α _feedback value of the first target image sub-region decays from 0.5 to 0.001 after m frames.

The first weight α _space of the second target image sub-region is taken as the position weight of the second target image sub-region, that is, α=α _space .

Further, after obtaining the images corresponding to the sub-region I _s α _i, for normalization, in order to achieve

Among them, α _i represents the weight of the ith image sub-region.

It should be noted that, in the embodiment of the present disclosure, it is not limited to use the above formula to set the above-mentioned second weight. For example, the minimum and maximum values of the second weight are set to multiple levels, and the Every time there is a frame of image between them, the value of the second weight is reduced by one level. Taking the maximum value of 0.5 and the minimum value of 0.1 as an example, when there is no interval image between the first image and the second image, the The second weight is set to 0.5, and when there is one frame of image between the first image and the second image, the second weight is set to 0.4 to decrease.

Further, when determining the second weight, it is not limited that the above content analysis result indicates that the second target image sub-region of the second image includes the target object. The second weights of the sub-regions of the image are all increased, otherwise, they may all be decreased, so as to increase or decrease the overall weight of the first image according to the content analysis result of the second image. Further, the above-mentioned second image may be an image in which the target object is found to exist.

Taking FIG. 10 as an example, an embodiment of the present disclosure may include a camera module, a video decoding module, a quality screening module, and a content analysis module, wherein:

The camera module can be built-in or external hardware that can collect images in real time;

The video decoding module can be used to decode the video stream from the camera module and convert it into an independent image of each frame;

The quality screening module is used to obtain the image quality score of the image, and the quality evaluation algorithm package used can be added and modified according to business needs, and one or more different quality evaluation algorithms can be used to perform regional independent statistics on the image. , a comprehensive summary;

The content analysis module is used to analyze the images that the quality screening module has judged to meet the picture quality, and output the analysis results. At the same time, the content analysis module feeds back the content analysis result of the target object found to the quality screening module, such as the region where the target object is located, and the quality screening module performs exponential attenuation of the image quality of the region within a certain subsequent frame interval. weighted.

In this way, the position weight of the image sub-region can be made more accurate through the feedback of the content analysis module.

Of course, in some scenes or for some images, the second weight may not be set, that is, only the first weight may be set. For example, if there is no target object in the multiple frames before the first image, the second weight may not be set.

As an optional implementation manner, the above calculation of the sub-region quality score of each image sub-region includes: for each image sub-region, using one or more image quality discrimination algorithms to discriminate the image quality of the image sub-region , get the sub-region quality score of the image sub-region.

And when multiple image quality judging algorithms are used, each image quality judging algorithm can set a corresponding weight.

For example: by

A total of j=1, 2,..., k image quality discrimination algorithms, Q _s represents the quality score of a certain image sub-region, f _j ( ) represents the initial quality score of the sub-region corresponding to the image quality discrimination algorithm, ω _j represents the weight corresponding to the image quality discrimination algorithm,

It should be noted that, in the embodiment of the present disclosure, the image quality determination algorithm includes, but is not limited to, the Laplacian algorithm and the Sobel algorithm.

In this embodiment, the image quality of the image sub-regions is discriminated according to at least one image quality discriminating algorithm, which can facilitate the upgrading of the image quality discriminating algorithm, and can discriminate against various indicators through different image quality discriminating algorithms, so that the The sub-region quality score can more accurately characterize the image quality of an image sub-region.

Further, in the embodiment of the present disclosure, the image quality discrimination algorithm for calculating the sub-region quality score of the sub-region of the image can also be modified or replaced according to the actual business requirements or the type of the image, so as to facilitate the realization of the premise of not changing the hardware of the electronic device in the later stage. Next, through the algorithm package upgrade, the discrimination of more picture discrimination indicators (such as brightness, color, etc.) is added.

In the embodiment of the present disclosure, since the sub-region quality scores of each image sub-region are determined respectively, the image quality of each image sub-region can be evaluated individually, and then the image of the first image is obtained according to the sub-region quality scores of each image sub-region The quality score enables the image quality score of the first image to more accurately reflect the image quality of the first image. Image quality scoring can be applied to image screening based on image quality, and can improve the accuracy of image screening. For example, images with unqualified image quality are filtered out, and content analysis is performed only on images with acceptable image quality, which can reduce the image content. The power consumption of analysis improves the efficiency of image content analysis.

Referring to FIG. 12, FIG. 12 is a flowchart of another image processing method provided by the implementation of the present disclosure, as shown in FIG. 12, including the following steps:

S601, decoding and image preprocessing are carried out to the video from the camera hardware of the electronic device, and the image I is obtained in a loop;

S602: Divide the I image into regions to obtain a total of s image sub-regions, and the resolution of each image sub-region is u×v. The adjacent image sub-regions in the s image sub-regions have overlapping regions. _{The first weight (or spatial weight) α space} is preset according to the positions of different image sub-regions in the image.

For example, the upper part of the image is the sky, which is of low importance, and the image near the lower part of the screen is the moving area of people to be monitored, which has a high degree of importance. Therefore, the spatial weight α _space can be determined according to business needs.

The quality screening module shown in Figure 10 receives the target position information fed back by the subsequent content analysis module, such as the target position information of the target region of interest (ROI, Region Of Interest) obtained by the subsequent Yolo algorithm, and then uses the exponential decay method to obtain the first _{The second weight α feedback of} a target image sub-region further obtains the position weight of the first target image sub-region α=α _space +α _feedback ; the position weight of the second target image sub-region α=α _space . The position weight α of each image sub-region is normalized, and the sum of the position weights of each image sub-region is equal to 1.

S603. For each image sub-region, use two algorithms, Laplacian and Sobel, to obtain the sub-region quality score of the image sub-region.

Specifically, two algorithms, Laplacian and Sobel, can be comprehensively used for ambiguity judgment. The weights between the two algorithms are ω ₁ and ω _{2 respectively} , and ω ₁ +ω ₂ =1.

Wherein, for each sub-region I _s:

Use the Laplacian algorithm to obtain the variance value of the image second derivative matrix as the initial sub-region quality score f ₁ (·); use the Sobel algorithm to obtain the mean value of the image difference matrix as the sub-region quality initial score f ₂ (·);

The sub-region quality score Q _s =ω ₁ ×f ₁ (·)+ω ₂ ×f ₂ (·) is obtained.

Afterwards, all the sub-regions of the image are summarized according to the position weights to obtain the image quality score Q _Total . Specifically, the sub-region quality scores of the sub-regions of the image may be multiplied by the corresponding position weights, and then summed.

S604. Perform threshold discrimination and decision on the image quality score.

This step may be to determine whether the frame of image enters the subsequent content analysis link according to the final image quality score Q _{Total, which may be specifically as follows:}

Wherein, T _Threshold represents a preset threshold.

In this way, content analysis can be performed on qualified images, and unqualified images are discarded, so as to reduce the power consumption of image content analysis.

Referring to FIG. 13, FIG. 13 is a structural diagram of an image processing apparatus provided by an embodiment of the present disclosure. As shown in FIG. 13, the image processing apparatus 100 includes:

The dividing module 101 is configured to divide the first image into regions to obtain a plurality of image sub-regions, wherein adjacent image sub-regions in the plurality of image sub-regions have overlapping regions;

The calculation module 102 is used to determine the sub-region quality score of each image sub-region, and the sub-region quality score represents the image quality of the image sub-region;

The acquiring module 103 is configured to determine the image quality score of the first image according to the sub-region quality scores of each image sub-region, where the image quality score represents the image quality of the first image.

As an optional implementation manner, the obtaining module 103 includes:

The image quality score obtaining unit is used for weighting the sub-region quality scores of each image sub-region according to the position weight of each image sub-region, and summing them up to obtain the image quality score of the first image, wherein the position weight of the image sub-region is Characterizes the importance of the position of the image sub-region in the first image.

As an optional implementation manner, the obtaining module 103 further includes:

The position weight determination unit is used for acquiring target position information, the target position information representing the position of the target object in the second image determined by performing content analysis on the second image, and the second image is an image collected before the first image.

The position weight determination unit is further configured to determine the position weight of each image sub-region according to the target position information.

As an optional implementation manner, the position weight determination unit is configured to divide the plurality of image sub-regions into at least one first target image sub-region and at least one second target image sub-region according to the target position information, and the first target image sub-region A region is an image sub-region in the first image that corresponds to the position of the target object in the second image.

The position weight determination unit is further configured to determine a second weight of each of the first target image sub-areas, where the second weight of the first target image sub-area represents the possibility that the first target image sub-area includes a target object.

The position weight determination unit is further configured to determine the position weight of each first target image sub-region according to the first weight of each first target image sub-region and the second weight of each first target image sub-region.

The position weight determination unit is further configured to determine the first weight of each second target image sub-region as the position weight of each second target image sub-region.

As an optional implementation manner, the position weight determination unit is further configured to perform normalization processing on the position weights of each image sub-region, so that the sum of the position weights of each image sub-region is equal to 1.

As an optional implementation manner, the position weight determination unit is further configured to determine the second weight of each sub-region of the first target image according to the time interval between the first image and the second image. The two weights are negatively related to the time interval between the first image and the second image.

As an optional implementation manner, the computing module 102 includes:

The sub-region quality judging unit is used for judging the image quality of the image sub-regions according to at least one image quality judging algorithm, and obtaining the sub-region quality initial scores corresponding to various image quality judging algorithms.

The sub-region quality scoring unit is configured to determine the sub-region quality scores of the image sub-regions according to the initial quality scores of each sub-region.

As an optional implementation manner, the image processing apparatus further includes a screening module.

The screening module is used to judge whether the image quality score is less than the preset threshold; in the case of the image quality score less than the preset threshold, discard the first image; in the case of the image quality score greater than or equal to the preset threshold, use the first image as the image to be analyzed.

Referring to FIG. 14, FIG. 14 is a structural diagram of an image processing apparatus provided by an embodiment of the present disclosure. As shown in FIG. 14, the image processing apparatus 200 includes:

An analysis module 201, configured to perform content analysis on the image to be analyzed to obtain a content analysis result;

The feedback module 202 is configured to generate target position information when the content analysis result indicates that the target object is included in the image to be analyzed, wherein the target position information represents the position of the target object in the image to be analyzed, and the target position information is used to determine the target position. The position weights of multiple image sub-regions in the first image are determined when the image quality of the first image is scored, the first image is an image collected after the image to be analyzed, and adjacent image sub-regions in the multiple image sub-regions overlap area.

It should be noted that, the image processing apparatus in the embodiments of the present disclosure may be an apparatus, and may also be a component, an integrated circuit, or a chip in an electronic device.

Referring to FIG. 15 , FIG. 15 is a structural diagram of an electronic device provided by an embodiment of the present disclosure. As shown in FIG. 15 , the electronic device 300 includes: a memory 301 , a processor 302 , and a memory 301 and a processor 302 A program or instruction running on the processor 302 implements the steps in the above image processing method when the program or instruction is executed by the processor 302 .

Embodiments of the present disclosure further provide a readable storage medium, where a program or an instruction is stored on the readable storage medium. When the program or instruction is executed by a processor, each process of the above image processing method embodiments can be implemented, and the same technology can be achieved. The effect, in order to avoid repetition, is not repeated here.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or apparatus that includes the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in the reverse order depending on the functions involved. To perform functions, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to some examples may be combined in other examples.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of this application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the inspiration of this application, without departing from the scope of protection of the purpose of this application and the claims, many forms can be made, which all fall within the protection of this application.

Claims

An image processing method, comprising:

Dividing the first image into regions to obtain a plurality of image sub-regions, wherein adjacent image sub-regions in the plurality of image sub-regions have overlapping regions;

determining a sub-region quality score of each of the image sub-regions, the sub-region quality score representing the image quality of the image sub-region;

The image quality score of the first image is determined according to the sub-region quality score of each of the image sub-regions, and the image quality score represents the image quality of the first image.
The method of claim 1, wherein the step of determining the image quality score of the first image according to the sub-region quality scores of each of the image sub-regions comprises:

The sub-region quality scores of each of the image sub-regions are weighted according to the position weight of each of the image sub-regions, and summed to obtain the image quality score of the first image, wherein the position weight of the image sub-regions is Indicates the importance of the position of the image sub-region in the first image.
The method according to claim 2, wherein, before the step of determining the image quality score of the first image according to the position weight and the sub-region quality score of each of the image sub-regions, according to each of the image sub-regions The step of determining the image quality score of the first image by the sub-area quality score further includes:

acquiring target position information, the target position information representing the position of the target object in the second image determined by performing content analysis on the second image, and the second image is an image collected before the first image;

The position weight of each of the image sub-regions is determined according to the target position information.
The method according to claim 3, wherein the step of determining the position weight of each of the image sub-regions according to the target position information comprises:

The plurality of image sub-regions are divided into at least one first target image sub-region and at least one second target image sub-region according to the target position information, and the first target image sub-region is in the first image an image sub-region corresponding to the position of the target object in the second image;

determining a second weight of each of the first target image sub-regions, where the second weight of the first target image sub-region represents the possibility that the first target image sub-region includes the target object;

Determine the position weight of each of the first target image sub-regions according to the first weight of each of the first target image sub-regions and the second weight of each of the first target image sub-regions;

Determining the first weight of each of the second target image sub-regions as the position weight of each of the second target image sub-regions;

The first weight of the image sub-region represents the importance of the position of the image sub-region in the first image.
The method according to claim 4, wherein the step of determining the position weight of each of the image sub-regions according to the target position information further comprises:

The position weights of each of the image sub-regions are normalized, so that the sum of the position weights of each of the image sub-regions is equal to 1.
The method of claim 4, wherein the step of determining the second weight of each of the first target image sub-regions comprises:

The second weight of each of the first target image sub-regions is determined according to the time interval between the first image and the second image, and the second weight of the first target image sub-region is related to the first image. is negatively correlated with the time interval between the second images.
The method according to any one of claims 1 to 6, wherein the step of determining the sub-region quality score of the image sub-region comprises:

Distinguish the image quality of the image sub-regions according to at least one image quality judging algorithm, and obtain the initial quality scores of the sub-regions corresponding to various image quality judging algorithms;

The sub-region quality score of the image sub-region is determined according to the initial quality score of each sub-region.
The method according to any one of claims 1 to 6, wherein after the step of determining the image quality score of the first image according to the sub-region quality scores of each of the image sub-regions, the method further comprises: :

judging whether the image quality score is less than a preset threshold;

In the case that the image quality score is less than the preset threshold, discarding the first image;

When the image quality score is greater than or equal to the preset threshold, the first image is used as the image to be analyzed.
An image processing method, comprising:

Perform content analysis on the image to be analyzed to obtain content analysis results;

When the content analysis result indicates that the image to be analyzed includes a target object, target position information is generated, wherein the target position information represents the position of the target object in the image to be analyzed, and the target The location information is used to determine the location weights of multiple image sub-regions in the first image when determining the image quality score of the first image, where the first image is an image collected after the image to be analyzed, and multiple The adjacent image sub-regions in the image sub-regions have overlapping regions.
An image processing device, comprising:

a dividing module, configured to divide the first image into regions to obtain a plurality of image sub-regions, wherein adjacent image sub-regions in the plurality of image sub-regions have overlapping regions;

a calculation module, configured to determine a sub-region quality score of each of the image sub-regions, where the sub-region quality score represents the image quality of the image sub-region;

An acquisition module, configured to determine an image quality score of the first image according to the sub-region quality scores of each of the image sub-regions, where the image quality score represents the image quality of the first image.
The apparatus of claim 10, wherein the acquiring module comprises:

an image quality score obtaining unit, configured to weight the sub-region quality scores of each of the image sub-regions according to the position weight of each of the image sub-regions, and sum them up to obtain the image quality score of the first image, wherein, The position weight of the image sub-region represents the importance of the position of the image sub-region in the first image.
The apparatus of claim 11, wherein the obtaining module further comprises:

The position weight determination unit is used for acquiring target position information, the target position information represents the position of the target object determined by performing content analysis on the second image in the second image, and the second image is in the second image. an image acquired before an image;

The position weight determination unit is further configured to determine the position weight of each of the image sub-regions according to the target position information.
The apparatus according to claim 12, wherein the position weight determination unit is configured to divide a plurality of the image sub-regions into at least one first target image sub-region and at least one second target image sub-region according to the target position information a target image sub-region, the first target image sub-region is an image sub-region corresponding to the position of the target object in the second image in the first image;

The position weight determination unit is further configured to determine the second weight of each of the first target image sub-regions, and the second weight of the first target image sub-region indicates that the first target image sub-region includes the target. the possibility of the object;

The position weight determination unit is further configured to determine the position of each of the first target image sub-regions according to the first weight of each of the first target image sub-regions and the second weight of each of the first target image sub-regions Weights;

The position weight determination unit is further configured to determine the first weight of each of the second target image sub-regions as the position weight of each of the second target image sub-regions;

The first weight of the image sub-region represents the importance of the position of the image sub-region in the first image.
The apparatus according to claim 13, wherein the position weight determination unit is further configured to normalize the position weight of each of the image sub-regions, so that the position weight of each of the image sub-regions has a and equal to 1.
The apparatus according to claim 13, wherein the position weight determination unit is further configured to determine, according to the time interval between the first image and the second image, the sub-regions of the first target image. The second weight, the second weight of the first target image sub-region is negatively correlated with the time interval between the first image and the second image.
The device according to any one of claims 10 to 15, wherein the computing module comprises:

a sub-area quality discrimination unit, configured to discriminate the image quality of the image sub-areas according to at least one image quality discrimination algorithm, and obtain initial sub-area quality scores corresponding to various image quality discrimination algorithms;

A sub-region quality scoring unit, configured to determine the sub-region quality scores of the image sub-regions according to the initial quality scores of each sub-region.
The device according to any one of claims 10 to 15, wherein the image processing device further comprises a screening module;

The screening module is configured to judge whether the image quality score is less than a preset threshold; in the case that the image quality score is less than the preset threshold, discard the first image; if the image quality score is greater than or equal to In the case of the preset threshold, the first image is used as the image to be analyzed.
An image processing device, comprising:

The analysis module is used to analyze the content of the image to be analyzed to obtain the content analysis result;

The feedback module is configured to generate target position information when the content analysis result indicates that the image to be analyzed includes a target object, wherein the target position information represents the position of the target object in the image to be analyzed. position, the target position information is used to determine the position weights of multiple image sub-regions in the first image when the image quality score of the first image is determined, and the first image is collected after the image to be analyzed image, and adjacent image sub-regions among the plurality of image sub-regions have overlapping regions.
An electronic device, characterized by comprising: a memory, a processor, and a program or instruction stored on the memory and executable on the processor, the program or instruction being executed by the processor to achieve the following At least one of the methods:

The image processing method according to any one of claims 1 to 8;

The image processing method according to claim 9.
A readable storage medium, characterized in that a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, at least one of the following methods is implemented:

The image processing method according to any one of claims 1 to 8;

The image processing method according to claim 9.