CN109672818B

CN109672818B - Method and device for adjusting image quality

Info

Publication number: CN109672818B
Application number: CN201710960703.2A
Authority: CN
Inventors: 潘积桂; 王宏博; 蔡金
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2017-10-16
Filing date: 2017-10-16
Publication date: 2020-12-22
Anticipated expiration: 2037-10-16
Also published as: CN109672818A

Abstract

The application discloses a method and a device for adjusting image quality, wherein the method comprises the following steps: respectively performing sampling processing on n images continuously obtained in the process of previewing the camera to obtain n sampling images, determining a group of motion areas according to every two adjacent sampling images in the n sampling images, and obtaining n-1 groups of motion areas; determining pixel displacement corresponding to the n-1 groups of motion areas according to the n-1 groups of motion areas; and determining the motion information corresponding to the n-1 groups of motion areas according to the pixel displacement corresponding to the n-1 groups of motion areas respectively. By the method, the image resolution can be reduced through sampling processing, the sampling image is obtained, the motion information is determined according to the sampling image, and the obtained motion information is used for adjusting the image quality. The method has the characteristics of low operation amount and low power consumption.

Description

Method and device for adjusting image quality

Technical Field

The present disclosure relates to the field of image processing and computer vision, and more particularly, to a method and apparatus for adjusting image quality.

Background

Some mobile electronic devices (e.g., mobile phones) are equipped with a camera, and a moving object may be included in a photographing scene, so that a moving smear of the object in an exposure time may appear as a blur phenomenon, which results in a low filming rate and affects a photographing effect. Therefore, it is very important to accurately detect the motion of the object, and the exposure parameters can be controlled based on accurate motion information of the object, thereby improving the imaging quality and the slice forming rate.

In the prior art, general motion detection schemes are divided into two categories: image processing based motion detection schemes and external microelectronic device based motion detection schemes.

For example, the motion detection scheme based on image processing includes a background method, an optical flow method, a time difference method, and the like. Among them, the background method is highly dependent on the scene and is not suitable for frequently changing scenes. The optical flow method is complex in calculation, and high in time consumption and power consumption. The time difference method is susceptible to noise.

The motion detection scheme based on the external microelectronic device depends on the displacement of the whole external electronic device sensing equipment, and the motion state of a shot object cannot be determined under the condition that the equipment is stable, so that the motion detection scheme is suitable for shooting anti-shake processing.

In summary, the existing motion detection schemes are not suitable for a camera in a mobile electronic device to capture a moving object.

Disclosure of Invention

The application provides a method and a device for adjusting image quality, which are used for providing accurate object motion information for a photographic camera in mobile electronic equipment.

In a first aspect, the present application provides a method for adjusting image quality, the method comprising:

the processor respectively executes sampling processing on n images continuously obtained in the process of previewing the camera to obtain n sampling images, wherein the ith image corresponds to the ith sampling image, the ith image is any one of the n images, and i and n are positive integers; determining a group of motion areas according to every two adjacent sampling images in the n sampling images, and obtaining n-1 groups of motion areas, wherein the ith group of motion areas is obtained according to the ith sampling image and the (i + 1) th sampling image, and the ith group of motion areas comprises the motion area corresponding to the ith sampling image and the motion area corresponding to the (i + 1) th sampling image; determining pixel displacement corresponding to the n-1 groups of motion areas according to the n-1 groups of motion areas, wherein the pixel displacement corresponding to the i group of motion areas refers to the pixel displacement of the motion area corresponding to the (i + 1) th sampling image relative to the motion area corresponding to the i sample image; and determining motion information corresponding to the n-1 groups of motion areas according to the pixel displacement corresponding to the n-1 groups of motion areas respectively, wherein the motion information corresponding to the n-1 groups of motion areas respectively is used for adjusting the image quality.

By the method, the processor can obtain n sampled images with lower image resolution by sampling the n images, determine a group of motion areas according to every two adjacent sampled images in the n sampled images, and further calculate the pixel displacement corresponding to the n-1 groups of motion areas respectively and the motion information corresponding to the n-1 groups of motion areas respectively. Therefore, the method can realize that the image resolution is reduced through sampling processing, the sampling image is obtained, the motion information is determined according to the sampling image, and the obtained motion information is used for adjusting the image quality. The method has the characteristics of low operation amount and low power consumption.

In one possible design, the ith sampled image is an image obtained by performing downsampling on the ith image after performing a preset filtering process in a horizontal direction of the ith image, and performing downsampling on the ith image in a vertical direction of the ith image.

Therefore, the method can reduce the computation amount and power consumption of the processor and effectively avoid the problem of sampling image flicker.

In one possible design, the predetermined filtering process is an IIR filter or a FIR filter.

Therefore, the application provides a plurality of possible preset filtering processing modes.

In one possible design, the processor may employ the following method in determining a set of motion regions from every two adjacent sampled images of the n sampled images:

for the ith sample image and the (i + 1) th sample image, executing: dividing the ith sampling image and the (i + 1) th sampling image into M multiplied by N rectangular windows, wherein M and N are positive integers; calculating M × N first correlation coefficients, wherein a kth first correlation coefficient is a correlation coefficient of a kth rectangular window in the ith sampling image and a kth rectangular window in the (i + 1) th sampling image, the kth first correlation coefficient is any one of the M × N first correlation coefficients, and k is a positive integer; and determining the ith group of motion areas according to rectangular windows corresponding to first correlation coefficients which are less than or equal to a first preset correlation coefficient threshold value in the M multiplied by N first correlation coefficients, wherein the rectangular windows included in the motion areas corresponding to the ith sampling image correspond to the rectangular windows included in the motion areas corresponding to the (i + 1) th sampling image in the ith group of motion areas in a one-to-one mode.

Therefore, through the method, the processor can determine that a moving object exists in the (i + 1) th sampling image relative to the ith sampling image and determine the area where the moving object exists.

In one possible design, the kth first correlation coefficient is a correlation coefficient of a projection histogram of a kth rectangular window in the ith sampling image in the horizontal direction and a projection histogram of a kth rectangular window in the i +1 th sampling image in the horizontal direction, and a correlation coefficient of a projection histogram of a kth rectangular window in the ith sampling image in the vertical direction and a projection histogram of a kth rectangular window in the i +1 th sampling image in the vertical direction.

Therefore, the correlation coefficient calculation method for calculating the first correlation coefficient may be a projection histogram method, an image histogram method, a color feature method, or the like, and the present application is not limited thereto. In order to reduce the computation amount and the power consumption of the processor, the processor can calculate the correlation coefficient of the two rectangular windows by adopting a projection histogram method.

In one possible design, the processor may employ the following method when determining the pixel displacement corresponding to each of the n-1 sets of motion areas according to the n-1 sets of motion areas:

for the ith group of motion areas, the motion area corresponding to the ith sampling image comprises m rectangular windows, and the motion area corresponding to the ith sampling image comprises m rectangular windows, executing: calculating m pixel displacements by adopting a preset algorithm, wherein the jth pixel displacement is the pixel displacement of a jth rectangular window in a motion region corresponding to the (i + 1) th sampling image, which is matched with the jth rectangular window in the motion region corresponding to the ith sampling image, relative to the jth rectangular window in the motion region corresponding to the ith sampling image, and m and j are positive integers; and determining pixel displacement corresponding to the ith group of motion areas according to the m pixel displacements.

Therefore, by the method, the processor can determine the speed and the direction of the movement of the object on the basis of lower computation amount and power consumption.

In one possible design, the motion information corresponding to the i-th group of motion areas includes at least one of the following: pixel displacement corresponding to the ith group of motion areas, motion speed identification corresponding to the ith group of motion areas and motion direction identification corresponding to the ith group of motion areas; the motion speed identifier corresponding to the ith group of motion areas is a motion speed identifier corresponding to a preset displacement threshold range in which the pixel displacement corresponding to the ith group of motion areas falls; the motion direction identifier corresponding to the ith group of motion areas is a motion direction identifier corresponding to a preset angle range in which the pixel displacement direction corresponding to the ith group of motion areas falls.

Thus, by the above method, the processor is able to output a variety of indicia indicative of how fast and in what direction the object is moving.

In a possible design, if the m pixel displacements satisfy at least one of the following first and second judgment criteria, the motion information corresponding to the ith group of motion areas includes a first identifier, and the first identifier is used to indicate that the ith group of motion areas is in a high-speed motion state; the first judgment criterion means that the number of the m second correlation coefficients which is less than or equal to a second preset correlation coefficient threshold value is greater than or equal to a first preset number; and the jth second correlation coefficient is the correlation coefficient of the jth rectangular window in the motion area corresponding to the i +1 th sampling image, which is matched with the jth rectangular window in the motion area corresponding to the i th sampling image, and the jth rectangular window in the motion area corresponding to the i th sampling image. The second judgment criterion is that the number of different directions in the m pixel displacement directions is greater than or equal to a second preset number.

Therefore, the method can make up the deficiency of calculation errors of the SAD scanning algorithm in a fast motion scene, and ensure the accuracy of the motion information of the object.

In one possible design, the processor performs low-pass filtering on the pixel displacements respectively corresponding to the n-1 sets of motion areas before determining the motion information respectively corresponding to the n-1 sets of motion areas.

Therefore, the method can eliminate the influence of random noise and singular value on the calculation result and ensure the stability of the output result.

In a possible design, after determining the motion information corresponding to the i-th group of motion areas, the processor adjusts the first threshold if the absolute value of the difference between the magnitude of the pixel displacement corresponding to the i-th group of motion areas and the first threshold is less than or equal to a fluctuation threshold.

Therefore, the method can eliminate the problem of frequent jitter of the motion speed identifier when the displacement of a plurality of pixels is near the threshold value, and ensure the stability of the output result.

In a second aspect, the present application provides an apparatus for adjusting image quality, the apparatus comprising:

the device comprises a sampling unit, a processing unit and a processing unit, wherein the sampling unit is used for respectively carrying out sampling processing on n images continuously obtained in the process of previewing a camera to obtain n sampling images, the ith image corresponds to the ith sampling image, the ith image is any one of the n images, and i and n are positive integers;

a processing unit, configured to determine a set of motion regions according to every two adjacent sample images in the n sample images, and obtain n-1 sets of motion regions, where an ith set of motion regions is obtained according to the ith sample image and an (i + 1) th sample image, and the ith set of motion regions includes a motion region corresponding to the ith sample image and a motion region corresponding to the (i + 1) th sample image;

a calculating unit, configured to determine, according to the n-1 sets of motion areas, pixel displacements corresponding to the n-1 sets of motion areas, respectively, where the pixel displacement corresponding to the i-th set of motion areas refers to a pixel displacement of a motion area corresponding to the (i + 1) -th sampling image relative to a motion area corresponding to the i-th sampling image;

and the analysis unit is used for determining the motion information corresponding to the n-1 groups of motion areas according to the pixel displacement corresponding to the n-1 groups of motion areas respectively, and the motion information corresponding to the n-1 groups of motion areas respectively is used for adjusting the image quality.

Optionally, the apparatus for adjusting image quality may also implement some or all of the optional implementations of the first aspect.

In a third aspect, the present application provides an apparatus for adjusting image quality, comprising: a memory for storing computer executable program code; a communication interface, and a processor coupled with the memory and the communication interface. Wherein the program code stored by the memory comprises instructions which, when executed by the processor, cause the apparatus for adjusting image quality to carry out the method of the first aspect as described above.

In a fourth aspect, the present application provides a computer program product comprising: computer program code for causing a computer to perform the method of any possible implementation of the first aspect described above, when the computer program code runs on a computer.

In a fifth aspect, the present application provides a computer-readable medium having program code stored thereon, which, when run on a computer, causes the computer to perform the method of the implementation of the first aspect described above.

In a sixth aspect, the present application provides a chip comprising: a processing module and a communication interface, the processing module being configured to perform the method of any possible implementation manner of the first aspect.

The chip further includes a storage module, the storage module is configured to store instructions, and the processing module is configured to execute the instructions stored by the storage module, and the execution of the instructions stored in the storage module causes the processing module to execute the method in any possible implementation manner in the first aspect.

Drawings

FIG. 1 is a flow chart of an overview of a method of adjusting image quality in the present application;

fig. 2(a) and 2(b) are schematic diagrams of the SAD scanning algorithm used in the present application to determine the rectangular window in the (i + 1) th image that matches the jth rectangular window in the ith sample image in the x-axis direction (horizontal direction);

FIG. 3 is a flowchart illustrating a process of determining motion information corresponding to two adjacent sampled images by a processor according to the present application;

FIG. 4 is a second flowchart illustrating the process of determining motion information corresponding to two adjacent sampled images by the processor;

FIG. 5 is a schematic structural diagram of an apparatus for adjusting image quality according to the present application;

fig. 6 is a schematic structural diagram of an apparatus for adjusting image quality according to the present application.

Detailed Description

Embodiments of the present application are described below with reference to the accompanying drawings.

The method for adjusting the image quality can be used for providing motion information for a photographic camera in the mobile electronic equipment. Further, the mobile electronic device may control an exposure parameter of the camera based on the motion information, for example, reduce the exposure parameter, or perform a compensation or correction process on the image by using an image post-processing algorithm in combination with the motion information, thereby improving the imaging quality and the filming rate. The method for adjusting image quality provided by the application can be executed by a processor or a chip in the mobile electronic device.

Referring to fig. 1, the present application provides a method for adjusting image quality, the method comprising:

step 100: the processor respectively executes sampling processing on n images continuously obtained in the process of previewing the camera to obtain n sampling images.

The n images can be small-scale pixel images, the ith image corresponds to the ith sampling image, the resolution of the ith sampling image is lower than that of the ith image, the ith image is any one of the n images, and i and n are positive integers.

For example, a photographic camera in a mobile electronic device may enable a succession of 30 images to be obtained per second during the camera preview process. The processor performs sampling processing according to the 30 images respectively, namely, sampling processing is performed on each image in the 30 images in sequence, and 30 sampled images are obtained.

In order to reduce the amount of computation and power consumption of the processor, reducing the image resolution is the most direct method, for example, the down-sampling process may be directly performed on each of n images, but n sampled images obtained by directly performing the down-sampling process on each of the n images may cause the sampled images to flicker, resulting in motion false detection. Therefore, the present application provides a possible sampling manner, taking the ith image as an example, and the sampling processing performed on the ith image specifically includes: downsampling is performed after a preset filtering process is performed in the horizontal direction of the ith image, and downsampling is performed in the vertical direction of the ith image.

In addition, the sampling process performed on the ith image may be: and performing down-sampling after the first preset filtering processing on the ith image in the horizontal direction, and performing down-sampling after the second preset filtering processing on the ith image in the vertical direction. The first preset filtering process and the second preset filtering process may be the same or different.

The above mentioned preset filtering process is an infinite impulse response filter (IIR filter) or a finite impulse response filter (FIR filter), which is not limited in this application.

By adopting the sampling mode, the problem of sampling image flicker can be effectively avoided for n obtained sampling images as long as the coefficient of the filtering processing is properly designed, and the calculation amount of the processor is low. For example, the total pixels of each sampled image is slightly below 50K.

Step 110: the processor determines a group of motion areas according to every two adjacent sampling images in the n sampling images, and obtains n-1 groups of motion areas.

The motion areas of the ith group are obtained according to the ith sampling image and the (i + 1) th sampling image, and the motion areas of the ith group comprise the motion areas corresponding to the ith sampling image and the motion areas corresponding to the (i + 1) th sampling image.

For example, assuming a total of 4 sampling images, every two adjacent sampling images determine a group of motion areas, i.e., the 1 st sampling image and the 2 nd sampling image determine the 1 st group of motion areas, the 2 nd sampling image and the 3 rd sampling image determine the 2 nd group of motion areas, and the 3 rd sampling image and the 4 th sampling image determine the 3 rd group of motion areas, so as to obtain 3 groups of motion areas.

The 1 st group of motion areas is obtained according to the 1 st sampling image and the 2 nd sampling image, and the 1 st group of motion areas comprises the motion area corresponding to the 1 st sampling image and the motion area corresponding to the 2 nd sampling image. The 2 nd group of motion areas is obtained according to the 2 nd sampling image and the 3 rd sampling image, and the 3 rd group of motion areas comprises the motion area corresponding to the 2 nd sampling image and the motion area corresponding to the 3 rd sampling image. The 3 rd group of motion areas are obtained according to the 3 rd sampling image and the 4 th sampling image, and the 3 rd group of motion areas comprise a motion area corresponding to the 3 rd sampling image and a motion area corresponding to the 4 th sampling image.

It should be understood that the motion region corresponding to the 2 nd sampling image included in the 1 st group of motion regions is different from the motion region corresponding to the 2 nd sampling image included in the 2 nd group of motion regions, because the motion region corresponding to the 2 nd sampling image included in the 1 st group of motion regions is obtained from the 1 st sampling image and the 2 nd sampling image, and refers to the motion region of the 2 nd sampling image relative to the 1 st sampling image in the 2 nd sampling image. And the motion area corresponding to the 2 nd sampling image included in the 2 nd group motion area is obtained according to the 2 nd sampling image and the 3 rd sampling image, and refers to the motion area of the 3 rd sampling image relative to the 2 nd sampling image in the 2 nd sampling image. Similarly, the motion region corresponding to the 3 rd sampling image included in the 2 nd group motion region is different from the motion region corresponding to the 3 rd sampling image included in the 3 rd group motion region.

Step 120: and the processor determines pixel displacement corresponding to the n-1 groups of motion areas according to the n-1 groups of motion areas.

The pixel displacement corresponding to the ith group of motion areas refers to the pixel displacement of the motion area corresponding to the (i + 1) th sampling image relative to the motion area corresponding to the ith sampling image.

Step 130: and the processor determines the motion information corresponding to the n-1 groups of motion areas according to the pixel displacement corresponding to the n-1 groups of motion areas respectively.

The motion information corresponding to the n-1 groups of motion areas is used for adjusting the image quality. For example, the processor may control exposure parameters of the photographic camera based on motion information corresponding to the n-1 sets of motion regions, respectively, for example, to reduce the exposure parameters to mitigate imaging blur caused by moving smear of the object during the exposure time. For another example, the processor may perform compensation or correction processing on the image by using an image post-processing algorithm in combination with the motion information corresponding to each of the n-1 sets of motion regions. Therefore, the image quality and the slice rate can be improved by using the motion information corresponding to the n-1 sets of motion areas respectively.

In steps 110 to 130, how to determine the i-th group motion region, the pixel displacement corresponding to the i-th group motion region, and the motion information corresponding to the i-th group motion region will be specifically described below by taking the i-th sample image and the i + 1-th sample image as examples.

First, the processor first determines the ith set of motion regions.

Aiming at the ith sampling image and the (i + 1) th sampling image, the processor executes:

(1) the processor divides the ith sampling image and the (i + 1) th sampling image into M multiplied by N rectangular windows, wherein M and N are positive integers.

It should be understood that each of the M × N rectangular windows in the ith sample image is equal in size and corresponds to each of the M × N rectangular windows in the ith sample image.

(2) And calculating M multiplied by N first correlation coefficients, wherein the kth first correlation coefficient is a correlation coefficient of a kth rectangular window in the ith sampling image and a kth rectangular window in the (i + 1) th sampling image, the kth first correlation coefficient is any one of the M multiplied by N first correlation coefficients, and k is a positive integer.

The correlation coefficient calculation method for calculating the first correlation coefficient may be a projection histogram method, an image histogram method, a color feature method, or the like, which is not limited in the present application.

In one possible design, to reduce the computation and power consumption of the processor, the processor may calculate the correlation coefficient of two rectangular windows using a projection histogram method. Therefore, the kth first correlation coefficient may be a correlation coefficient of a projection histogram of the kth rectangular window in the horizontal direction in the ith sample image and a projection histogram of the kth rectangular window in the i +1 th sample image in the horizontal direction, and a correlation coefficient of a projection histogram of the kth rectangular window in the ith sample image in the vertical direction and a projection histogram of the kth rectangular window in the i +1 th sample image in the vertical direction.

(3) If the kth first correlation coefficient is less than or equal to the first preset correlation coefficient threshold, that is, the kth rectangular window in the ith sampling image has a larger difference from the kth rectangular window in the (i + 1) th sampling image, it may be determined that the kth rectangular window is a motion window, and therefore, the processor determines that the motion region corresponding to the ith sampling image includes the kth rectangular window in the ith sampling image, and the motion region corresponding to the (i + 1) th sampling image includes the kth rectangular window in the (i + 1) th sampling image.

By adopting the method, the processor traverses M multiplied by N first correlation coefficients, compares each first correlation coefficient with the first correlation coefficient, and takes all rectangular windows corresponding to all the first correlation coefficients which are less than or equal to a first preset correlation coefficient threshold value as motion windows respectively so as to determine the ith group of motion areas, wherein the rectangular windows included in the motion areas corresponding to the ith sampling image correspond to the rectangular windows included in the motion areas corresponding to the (i + 1) th sampling image in the ith group of motion areas in a one-to-one correspondence manner.

Second, the processor determines the pixel displacement corresponding to the ith set of motion regions.

For the ith group of motion areas, the motion area corresponding to the ith sampling image comprises m rectangular windows, and the processor executes:

(1) the processor calculates m pixel displacements by adopting a preset algorithm, wherein the jth pixel displacement is the pixel displacement of a jth rectangular window in a motion area corresponding to the (i + 1) th sampling image, which is matched with the jth rectangular window in the motion area corresponding to the ith sampling image, relative to a jth rectangular window in the motion area corresponding to the ith sampling image, and m and j are positive integers.

The preset algorithm may be a SAD scanning algorithm, or other algorithms, which is not limited in this application.

As shown in fig. 2(a) and 2(b), a schematic diagram of determining a rectangular window matching the jth rectangular window in the ith sample image in the (i + 1) th image in the x-axis direction (horizontal direction) by using the SAD scanning algorithm is shown. Fig. 2(a) is a projection histogram of the jth rectangular window in the ith sample image in the x-axis direction. Fig. 2(b) is a projection histogram in the x-axis direction corresponding to the (j-1) th rectangular window to the (j + 1) th rectangular window in the (i + 1) th sampled image. The dashed frame is a scan frame, and the size of the scan frame is the same as that of the rectangular frame, and the rectangular frame defined by the dashed frame in fig. 2(b) is a rectangular window matched with the jth rectangular window in the ith sample image in the (i + 1) th image in the x-axis direction. Therefore, the pixel displacement component of the jth rectangular window in the ith sample image in the x-axis direction with respect to the (i + 1) th image can be determined from the dashed line box.

(2) And the processor determines the pixel displacement corresponding to the ith group of motion areas according to the m pixel displacements.

And obtaining the average value of pixel displacement components in the x-axis direction and the average value of pixel displacement components in the y-axis direction according to the m pixel displacements, and further determining the pixel displacement corresponding to the i-th group of motion areas.

Second, the processor determines motion information corresponding to the ith set of motion regions.

The motion information corresponding to the ith group of motion areas comprises at least one of the following: the motion speed identification and the motion direction identification are respectively corresponding to the motion areas of the ith group.

And the motion speed identifier corresponding to the ith group of motion areas is the motion speed identifier corresponding to a preset displacement threshold range in which the pixel displacement corresponding to the ith group of motion areas falls.

For example, 1 pixel to 5 pixels correspond to the motion velocity flag 1, 5 pixels to 8 pixels correspond to the motion velocity flag 2, and 8 pixels or more correspond to the motion velocity flag 3. Assuming that the magnitude of the pixel displacement corresponding to the ith group of motion areas is 3, the motion speed identifier corresponding to the ith group of motion areas is a motion speed identifier 1.

The motion direction identifier corresponding to the ith group of motion areas is a motion direction identifier corresponding to a preset angle range in which the pixel displacement direction corresponding to the ith group of motion areas falls.

For example, 360 degrees is divided into 12 angle intervals, each 30 degrees corresponds to one motion direction identifier, and the total number of the motion direction identifiers is 12. Assuming that the direction of pixel displacement corresponding to the ith group of motion areas is 45 degrees, the motion speed identifier corresponding to the ith group of motion areas is motion speed identifier 2.

Therefore, the method and the device can provide accurate object motion information, and can determine the motion speed and the motion direction.

Further, in order to ensure the accuracy of the above result, the scanning range corresponding to the SAD scanning algorithm is considered, for example, the scanning range is generally 3 to 5 rectangular windows, and therefore, when the object moving speed is fast and exceeds the scanning range, the motion information may be inaccurate. The application provides a method for judging whether an object moves at a high speed.

Specifically, after the processor calculates m pixel displacements by using the SAD scanning algorithm, if the m pixel displacements satisfy at least one of the following first and second judgment criteria, the motion information corresponding to the i-th group of motion regions includes a first identifier, and the first identifier is used to indicate that the i-th group of motion regions is in a high-speed motion state.

The first judgment criterion refers to that the number of the m second correlation coefficients which is less than or equal to a second preset correlation coefficient threshold value is greater than or equal to a first preset number;

and the jth second correlation coefficient is the correlation coefficient of the jth rectangular window in the motion area corresponding to the (i + 1) th sampling image, which is matched with the jth rectangular window in the motion area corresponding to the ith sampling image, and the jth rectangular window in the motion area corresponding to the ith sampling image.

It should be understood that the algorithm of the second correlation coefficient may be the same as or different from the algorithm of the first correlation coefficient, and the second preset correlation coefficient threshold may be the same as or different from the first preset correlation coefficient threshold, and the repeated description is omitted here.

The second judgment criterion is that the number of different directions in the displacement directions of the m pixels is greater than or equal to a second preset number.

For example, when m is 10, the number of the 10 second correlation coefficients that is less than or equal to the second preset correlation coefficient threshold is 6 or more than or equal to the first preset number 5, and the number of different directions in the 10 pixel displacement directions is 7 or more than or equal to the second preset number 6, it may be determined that the ith group of motion regions is in a high-speed motion state, the motion information corresponding to the ith group of motion regions includes the first identifier, and no other information may be carried at this time.

Therefore, the judgment can make up the deficiency of calculation errors of the SAD scanning algorithm in a fast motion scene, and ensure the accuracy of the motion information of the object.

In addition, to ensure the stability of the above results, in one possible design, the processor may perform low-pass filtering on the pixel displacements corresponding to the n-1 sets of motion regions to eliminate the influence of random noise and singular values on the calculation results before determining the motion information corresponding to the n-1 sets of motion regions.

In another possible design, after determining the motion information corresponding to each of the ith group of motion areas, if the absolute value of the difference between the magnitude of the pixel displacement corresponding to the ith group of motion areas and the first threshold is less than or equal to the fluctuation threshold, the processor may adjust the first threshold, so as to eliminate the problem of frequent jitter of the motion speed identifier when the magnitudes of the pixel displacements are near the threshold.

For example, 1 pixel to 5 pixels are initially set to correspond to the motion speed identifier 1, 5 pixels to 8 pixels are set to correspond to the motion speed identifier 2, the fluctuation threshold is 1, the size of the pixel displacement corresponding to the 1 st group of motion areas is 4 pixels, 4 is less than 5, and the 1 st group of motion areas corresponds to the motion speed identifier 1. The absolute value of the difference between the pixel displacement corresponding to the motion area in the group 1 and the upper threshold of the motion speed identifier 1 is 1, and 5-4 is equal to 1, and at this time, the absolute value of the difference is equal to the fluctuation threshold 1, and then the processor adjusts the upper threshold of the motion speed identifier 1 to 5+1, which is equal to 6. The pixel displacement corresponding to the 2 nd group motion area is 5 pixels, 5 is less than 6, the motion speed identifier corresponding to the 2 nd group motion area is the motion speed identifier 1, the absolute value of the difference between the pixel displacement corresponding to the 2 nd group motion area and the upper threshold of the latest motion speed identifier 1 is 1, 6-5 is 1, and at this time, the absolute value of the difference is equal to the fluctuation threshold 1, and then the processor adjusts the upper threshold of the motion speed identifier 1 to be 6+1 is 7. The pixel displacement corresponding to the 3 rd group motion area is 5 pixels, 5 is less than 7, and the motion speed mark corresponding to the 3 rd group motion area is the motion speed mark 1. The absolute value of the difference between the pixel displacement corresponding to the 3 rd group motion area and the upper threshold of the latest motion speed identifier 1 is 2, and 7-5 is 2, at this time, the absolute value of the difference is greater than the fluctuation threshold 1, the processor does not adjust the upper threshold of the motion speed identifier 1, and the upper threshold of the motion speed identifier 1 is maintained at 7.

For another example, 1 pixel to 5 pixels are initially set to correspond to the motion speed identifier 1, 5 pixels to 8 pixels are set to correspond to the motion speed identifier 2, the fluctuation threshold is 1, the size of the pixel displacement corresponding to the 1 st group of motion areas is 6 pixels, 5 is less than 6, and the 1 st group of motion areas corresponds to the motion speed identifier 2. The absolute value of the difference between the pixel displacement corresponding to the motion area in the 1 st group and the upper threshold of the motion speed identifier 1 is 1, 6-5 is 1, and at this time, the absolute value of the difference is equal to the fluctuation threshold 1, and then the processor adjusts the upper threshold of the motion speed identifier 1 to 5-1 is 4. In addition, the absolute value of the difference between the pixel displacement corresponding to the motion area group 1 and the upper threshold of the motion speed identifier 2 is 2, and at this time, the absolute value of the difference is greater than the fluctuation threshold 1, and then the processor does not adjust the upper threshold of the motion speed identifier 2. The pixel displacement corresponding to the 2 nd group motion area is 5 pixels, 4 is less than 5, the motion speed identifier corresponding to the 2 nd group motion area is the motion speed identifier 2, the absolute value of the difference between the pixel displacement corresponding to the 2 nd group motion area and the upper threshold of the latest motion speed identifier 1 is 1, 5-4 is 1, and at this time, the absolute value of the difference is equal to the fluctuation threshold 1, and then the processor adjusts the upper threshold of the motion speed identifier 1 to be 4-1-3. The pixel displacement corresponding to the 3 rd group motion area is 5 pixels, 3 is less than 5, and the motion speed mark corresponding to the 3 rd group motion area is the motion speed mark 2. The absolute value of the difference between the pixel displacement corresponding to the 3 rd group motion area and the upper threshold of the latest motion speed identifier 1 is 2, and 5-3 is 2, and at this time, the absolute value of the difference is greater than the fluctuation threshold 1, the processor does not adjust the upper threshold of the motion speed identifier 1, and the upper threshold of the motion speed identifier 1 is maintained at 3.

Therefore, the frequent jitter of the movement speed identifier, which occurs when the displacement of a plurality of pixels is close to the threshold value, can be eliminated by dynamically adjusting the threshold value, and the stability of the output result is ensured.

Referring to fig. 3, a specific flowchart of the processor determining motion information corresponding to two adjacent sampled images is shown.

S301: two adjacent sampling images are respectively divided into M multiplied by N rectangular windows, and projection histograms in the x-axis direction and the y-axis direction are respectively made for each rectangular window in each sampling image.

S302: m × N first correlation coefficients are calculated.

Wherein, each first correlation coefficient is the sum of the correlation coefficient of the projection histogram of the rectangular window at the same position in the two adjacent sampling images in the horizontal direction and the correlation coefficient of the projection histogram in the vertical direction.

S303: determining whether the current rectangular window is a moving window, if the first correlation coefficient corresponding to the current rectangular window is less than or equal to the first preset correlation coefficient threshold, executing S304, and if the first correlation coefficient corresponding to the current rectangular window is greater than the first preset correlation coefficient threshold, executing S305.

S304: and calculating the pixel displacement corresponding to the current rectangular window.

S305: and judging whether the current rectangular window is the last rectangular window, if so, executing 306, otherwise, moving the rectangular window indicated by the current rectangular window to the next rectangular window, and returning to the step S303.

S306: and synthesizing pixel displacement corresponding to all the motion windows respectively and outputting the motion speed identifications and the corresponding motion direction identifications corresponding to the two adjacent sampling images.

By the embodiment shown in fig. 3, the processor can detect the motion of the object with low computation amount and low power consumption, and determine the motion information corresponding to two adjacent sampling images.

Referring to fig. 4, a second specific flowchart of the processor determining motion information corresponding to two adjacent sampled images is shown.

S401: two adjacent sampling images are respectively divided into M multiplied by N rectangular windows.

S402: the number of motion windows is determined to be m, and m pixel displacements.

Wherein m is a positive integer.

S403: m second correlation coefficients are calculated.

S404: and counting the number of different directions in the displacement directions of the m pixels.

S405: and judging whether the number of the m second correlation coefficients which is less than or equal to a second preset correlation coefficient threshold value is greater than or equal to a first preset number or not and whether the number of different directions in the m pixel displacement directions is greater than or equal to a second preset number or not, if so, executing S406, otherwise, executing S407.

S406: and outputting the first identifier and ending the flow.

S407: and outputting the corresponding movement speed identifications and the corresponding movement direction identifications of the two adjacent sampling images by integrating the displacement of the m pixels.

Therefore, by using the embodiment shown in fig. 4, the result obtained by using the embodiment shown in fig. 3 can be verified, so that the defect of calculation error of the SAD scanning algorithm in a fast motion scene is overcome, and the accuracy of the object motion information is ensured.

Referring to fig. 5, the present application provides an apparatus 500 for adjusting image quality, the apparatus comprising:

the sampling unit 501 is configured to perform sampling processing on n images continuously obtained in a camera preview process to obtain n sampled images, where an ith image corresponds to the ith sampled image, the ith image is any one of the n images, and i and n are positive integers;

a processing unit 502, configured to determine a set of motion regions according to every two adjacent sampled images in the n sampled images, and obtain n-1 sets of motion regions, where an ith set of motion regions is obtained according to the ith sampled image and an (i + 1) th sampled image, and the ith set of motion regions includes a motion region corresponding to the ith sampled image and a motion region corresponding to the (i + 1) th sampled image;

a calculating unit 503, configured to determine, according to the n-1 sets of motion areas, pixel displacements corresponding to the n-1 sets of motion areas, respectively, where the pixel displacement corresponding to the i-th set of motion areas refers to a pixel displacement of a motion area corresponding to the (i + 1) th sampled image relative to a motion area corresponding to the i-th sampled image;

an analyzing unit 504, configured to determine, according to pixel displacements respectively corresponding to the n-1 sets of motion regions, motion information respectively corresponding to the n-1 sets of motion regions, where the motion information respectively corresponding to the n-1 sets of motion regions is used to adjust image quality.

In a possible design, when determining a group of motion regions according to every two adjacent sampled images in the n sampled images, the processing unit 502 is specifically configured to:

for the ith sample image and the (i + 1) th sample image, executing:

dividing the ith sampling image and the (i + 1) th sampling image into M multiplied by N rectangular windows, wherein M and N are positive integers;

calculating M × N first correlation coefficients, wherein a kth first correlation coefficient is a correlation coefficient of a kth rectangular window in the ith sampling image and a kth rectangular window in the (i + 1) th sampling image, the kth first correlation coefficient is any one of the M × N first correlation coefficients, and k is a positive integer;

and determining the ith group of motion areas according to rectangular windows corresponding to first correlation coefficients which are less than or equal to a first preset correlation coefficient threshold value in the M multiplied by N first correlation coefficients, wherein the rectangular windows included in the motion areas corresponding to the ith sampling image correspond to the rectangular windows included in the motion areas corresponding to the (i + 1) th sampling image in the ith group of motion areas in a one-to-one mode.

In a possible design, when determining pixel displacements corresponding to the n-1 sets of motion regions according to the n-1 sets of motion regions, the calculating unit 503 is specifically configured to:

for the ith group of motion areas, the motion area corresponding to the ith sampling image comprises m rectangular windows, and the motion area corresponding to the (i + 1) th sampling image comprises m rectangular windows, executing:

calculating m pixel displacements by adopting a preset algorithm, wherein the jth pixel displacement is the pixel displacement of a jth rectangular window in a motion region corresponding to the (i + 1) th sampling image, which is matched with the jth rectangular window in the motion region corresponding to the ith sampling image, relative to the jth rectangular window in the motion region corresponding to the ith sampling image, and m and j are positive integers;

and determining pixel displacement corresponding to the ith group of motion areas according to the m pixel displacements.

In one possible design, the motion information corresponding to the i-th group of motion areas includes at least one of the following: pixel displacement corresponding to the ith group of motion areas, motion speed identification corresponding to the ith group of motion areas and motion direction identification corresponding to the ith group of motion areas;

the motion speed identifier corresponding to the ith group of motion areas is a motion speed identifier corresponding to a preset displacement threshold range in which the pixel displacement corresponding to the ith group of motion areas falls;

In a possible design, if the m pixel displacements satisfy at least one of the following first and second judgment criteria, the motion information corresponding to the ith group of motion areas includes a first identifier, and the first identifier is used to indicate that the ith group of motion areas is in a high-speed motion state;

the first judgment criterion means that the number of the m second correlation coefficients which is less than or equal to a second preset correlation coefficient threshold value is greater than or equal to a first preset number;

and the jth second correlation coefficient is the correlation coefficient of the jth rectangular window in the motion area corresponding to the i +1 th sampling image, which is matched with the jth rectangular window in the motion area corresponding to the i th sampling image, and the jth rectangular window in the motion area corresponding to the i th sampling image.

The second judgment criterion is that the number of different directions in the m pixel displacement directions is greater than or equal to a second preset number.

In one possible design, before determining the motion information corresponding to each of the n-1 sets of motion areas, the analysis unit 504 is further configured to:

and performing low-pass filtering on the pixel displacement corresponding to the n-1 groups of motion areas respectively.

In one possible design, after determining the motion information corresponding to the n-1 sets of motion areas, respectively, the analysis unit 504 is further configured to:

and if the absolute value of the difference value between the pixel displacement corresponding to the n-1 groups of motion areas and a first threshold value is less than or equal to a fluctuation threshold value, adjusting the first threshold value.

It should be understood that the above division of each unit is only a division of a logical function, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these units can be implemented entirely in software, invoked by a processing element; or may be implemented entirely in hardware; part of the units can also be realized in the form of software called by a processing element, and part of the units can be realized in the form of hardware. For example, the processing unit may be a processing element separately set up, or may be implemented by being integrated in a certain chip, or may be stored in a memory in the form of a program, and a certain processing element calls and executes the function of the unit. The other units are implemented similarly. In addition, all or part of the units can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, the steps of the method or the units above may be implemented by hardware integrated logic circuits in a processor element or instructions in software.

Based on the above embodiment, referring to fig. 6, an embodiment of the present application further provides an apparatus 600 for adjusting image quality, where the apparatus 600 for adjusting image quality includes: a communication interface 601, a processor 602, and a memory 603. The n images obtained continuously are obtained through the communication interface 601, the motion information determined by the analysis unit can be output through the communication interface 601, and the functions of the sampling unit, the processing unit, the calculation unit and the analysis unit can be realized through the processor 602.

The memory 603 is used for storing computer program code. The Memory 603 may include a Random Access Memory (RAM) or the like, and may also include a nonvolatile Memory, such as at least one disk Memory. The processor 602 may be a Central Processing Unit (CPU), a Network Processor (NP), a hardware chip, or any combination thereof. The processor 602 executes the computer program code stored in the memory 603 to implement the method shown in fig. 1.

In summary, according to the method for adjusting image quality provided by the present application, a processor performs sampling processing on n images continuously obtained in a camera preview process, respectively, to obtain n sampled images. Obtaining a low-resolution sampling image through sampling processing, determining a group of motion areas by a processor according to every two adjacent sampling images in the n sampling images to obtain n-1 groups of motion areas, namely identifying the motion areas, determining pixel displacement corresponding to the n-1 groups of motion areas respectively according to the n-1 groups of motion areas, determining motion information corresponding to the n-1 groups of motion areas respectively according to the pixel displacement corresponding to the n-1 groups of motion areas respectively, and using the motion information corresponding to the n-1 groups of motion areas respectively to adjust the image quality.

Therefore, the method can realize that the image resolution is reduced through sampling processing, the sampling image is obtained, the motion information is determined according to the sampling image, and the obtained motion information is used for adjusting the image quality. The method has the characteristics of low operation amount and low power consumption.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications may be made in the embodiments of the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to encompass such modifications and variations.

Claims

1. A method for adjusting image quality, the method comprising:

respectively performing sampling processing on n images continuously obtained in a camera previewing process to obtain n sampled images, wherein the ith image corresponds to the ith sampled image, the resolution of the ith sampled image is lower than that of the ith image, the ith image is any one of the n images, and i and n are positive integers; the ith sampling image is an image obtained by performing down-sampling on the ith image after the ith image is subjected to preset filtering processing in the horizontal direction and performing down-sampling in the vertical direction of the ith image;

determining a group of motion areas according to every two adjacent sampling images in the n sampling images to obtain n-1 groups of motion areas, wherein the ith group of motion areas is obtained according to the ith sampling image and the (i + 1) th sampling image, and the ith group of motion areas comprises the motion area corresponding to the ith sampling image and the motion area corresponding to the (i + 1) th sampling image;

determining pixel displacement corresponding to the n-1 groups of motion areas according to the n-1 groups of motion areas, wherein the pixel displacement corresponding to the i group of motion areas refers to the pixel displacement of the motion area corresponding to the (i + 1) th sampling image relative to the motion area corresponding to the i sample image;

and determining motion information corresponding to the n-1 groups of motion areas according to the pixel displacement corresponding to the n-1 groups of motion areas respectively, wherein the motion information corresponding to the n-1 groups of motion areas respectively is used for adjusting the image quality.

2. The method of claim 1, wherein the predetermined filtering process is an Infinite Impulse Response (IIR) filter or a Finite Impulse Response (FIR) filter.

3. The method of claim 1 or 2, wherein determining a set of motion regions from every two adjacent ones of the n sampled images comprises:

for the ith sample image and the (i + 1) th sample image, executing:

4. The method according to claim 3, wherein the kth first correlation coefficient is a sum of a correlation coefficient of a projection histogram of a kth rectangular window in the i-th sampled image in the horizontal direction and a projection histogram of a kth rectangular window in the i + 1-th sampled image in the horizontal direction, and a correlation coefficient of a projection histogram of a kth rectangular window in the i-th sampled image in the vertical direction and a projection histogram of a kth rectangular window in the i + 1-th sampled image in the vertical direction.

5. The method of claim 3, wherein determining the pixel displacement corresponding to each of the n-1 sets of motion regions according to the n-1 sets of motion regions comprises:

6. The method of claim 5, wherein the motion information corresponding to the i-th group of motion areas comprises at least one of: pixel displacement corresponding to the ith group of motion areas, motion speed identification corresponding to the ith group of motion areas and motion direction identification corresponding to the ith group of motion areas;

7. The method according to claim 5, wherein if the m pixel displacements satisfy at least one of the following first and second determination criteria, the motion information corresponding to the i-th group of motion areas includes a first flag indicating that the i-th group of motion areas is in a high-speed motion state;

wherein, the jth second correlation coefficient is a correlation coefficient of a jth rectangular window in a motion region corresponding to the i +1 th sampling image, which is matched with the jth rectangular window in the motion region corresponding to the i th sampling image, and a jth rectangular window in the motion region corresponding to the i th sampling image;

8. The method according to claim 1 or 2, wherein before determining the motion information corresponding to the n-1 sets of motion areas, respectively, further comprising:

9. The method according to claim 1 or 2, wherein after determining the motion information corresponding to the i-th group of motion areas, further comprising:

and if the absolute value of the difference value between the pixel displacement corresponding to the i groups of motion areas and a first threshold value is less than or equal to a fluctuation threshold value, adjusting the first threshold value.

10. An apparatus for adjusting image quality, the apparatus comprising:

the device comprises a sampling unit, a processing unit and a processing unit, wherein the sampling unit is used for respectively carrying out sampling processing on n images continuously obtained in the process of previewing a camera to obtain n sampled images, the ith image corresponds to the ith sampled image, the resolution of the ith sampled image is lower than that of the ith image, the ith image is any one of the n images, and i and n are positive integers; the ith sampling image is an image obtained by performing down-sampling on the ith image after the ith image is subjected to preset filtering processing in the horizontal direction and performing down-sampling in the vertical direction of the ith image;

the processing unit is used for determining a group of motion areas according to every two adjacent sampling images in the n sampling images to obtain n-1 groups of motion areas, wherein the ith group of motion areas is obtained according to the ith sampling image and the (i + 1) th sampling image, and the ith group of motion areas comprises the motion area corresponding to the ith sampling image and the motion area corresponding to the (i + 1) th sampling image;

11. The apparatus of claim 10, wherein the predetermined filtering process is an Infinite Impulse Response (IIR) filter or a Finite Impulse Response (FIR) filter.

12. The apparatus according to claim 10 or 11, wherein when determining a group of motion regions from every two adjacent sampled images of the n sampled images, the processing unit is specifically configured to:

for the ith sample image and the (i + 1) th sample image, executing:

13. The apparatus according to claim 12, wherein the kth first correlation coefficient is a sum of a correlation coefficient of a projection histogram of a kth rectangular window in the i-th sampled image in a horizontal direction and a projection histogram of a kth rectangular window in the i + 1-th sampled image in the horizontal direction, and a correlation coefficient of a projection histogram of a kth rectangular window in the i-th sampled image in a vertical direction and a projection histogram of a kth rectangular window in the i + 1-th sampled image in the vertical direction.

14. The apparatus according to claim 12, wherein when determining, according to the n-1 sets of motion regions, pixel displacements corresponding to the n-1 sets of motion regions, the computing unit is specifically configured to:

15. The apparatus of claim 14, wherein the motion information corresponding to the i-th group of motion regions comprises at least one of: pixel displacement corresponding to the ith group of motion areas, motion speed identification corresponding to the ith group of motion areas and motion direction identification corresponding to the ith group of motion areas;

16. The apparatus according to claim 14, wherein if the m pixel displacements satisfy at least one of the following first and second determination criteria, the motion information corresponding to the i-th group of motion areas includes a first flag indicating that the i-th group of motion areas is in a high-speed motion state;

17. The apparatus according to claim 10 or 11, wherein, before determining the motion information corresponding to the n-1 sets of motion areas, respectively, the analyzing unit is further configured to:

18. The apparatus according to claim 10 or 11, wherein after determining the motion information corresponding to the i-th group of motion areas, the analyzing unit is further configured to: