CN105335987B - Image processing method and device - Google Patents
Image processing method and device Download PDFInfo
- Publication number
- CN105335987B CN105335987B CN201410253811.2A CN201410253811A CN105335987B CN 105335987 B CN105335987 B CN 105335987B CN 201410253811 A CN201410253811 A CN 201410253811A CN 105335987 B CN105335987 B CN 105335987B
- Authority
- CN
- China
- Prior art keywords
- image block
- image
- intermediate image
- row
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Compression Of Band Width Or Redundancy In Fax (AREA)
- Image Processing (AREA)
Abstract
A kind of image processing method and device are disclosed, applied to the first electronic equipment, methods described includes：The first intermediate image block is extracted in the first image, described first image includes M row N row pixels, and the first intermediate image block includes I row J row pixels, M >=I, N >=J；Calculate the often pixel value sum of row pixel, and store I the first values in the first intermediate image block；Stored I the first values are utilized, calculate in the first intermediate image block the pixel value average value of each first image block in (I m+1) individual first image block successively, wherein described first image block includes m row J row pixels, I>m.It is more than the first intermediate image block of the image block for correlation calculations by being extracted in the first image, data analysis calculating is carried out to the first intermediate image block and preserves results of intermediate calculations, results of intermediate calculations is used in the correlation calculations of multiple first image blocks included in the first intermediate image block, realizes the multiplexing of computing resource.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image data processing method and apparatus.
Background
In image processing such as image encoding, image decoding, moving image analysis, and the like, it is generally necessary to calculate the correlation between image blocks in two images. In calculating the correlation between image blocks in two images, a fixed-size image block is generally selected as a calculation target. As shown in fig. 1, a first image block in a first image and a second image block in a second image are shown, both the first image block and the second image block in fig. 1 being 9 x 9 pixel image blocks.
Taking the example of calculating the correlation between a first image block in a first image and a second image block in a second image, the correlation between the first image block and the second image block may be calculated according to the following formula,
wherein the content of the first and second substances,
A＝a(lia+i-1)(lja+j-1)
B＝b(lib+i-1)(ljb+j-1)
specifically, a (lia + i-1) (lja + j-1) represents the ith row and jth column pixel value in the first image block, b (lib + i-1) (ljb + j-1) represents the ith row and jth column pixel value in the second image block, where lia is the row number of the first row of the first image block in the first image, lja is the column number of the first column of the first image block in the first image, lib is the row number of the first row of the second image block in the second image, and ljb is the column number of the first column of the second image block in the second image.
As can be seen from the above equation, when calculating the correlation between the first image block in the first image and the second image block in the second image, it is necessary to calculate the mean pixel value of the first image block, the mean pixel value of the second image block, the variance pixel value of the first image block, the variance pixel value of the second image block, and the product of the pixel values of the first image block and the second image block.
Then, when calculating the correlation between the first image block in the first image and the third image block in the second image, it is necessary to calculate the mean pixel value of the third image block, the variance pixel value of the third image block, and the product pixel value of the third image block and the first image block. Even if the second image block differs from the third image block only one row above or below, some intermediate results for the second image block are not recycled.
In the calculation process, a large amount of hardware resources and software resources of the image processing device are occupied, and intermediate results generated in each correlation calculation are not fully utilized, so that the resource utilization rate is low.
Therefore, a new image data processing method and apparatus are needed, which can make full use of hardware resources and software resources in the image processing device, so that the resource utilization rate is greatly improved.
Disclosure of Invention
In order to solve the above technical problem, the present invention provides an image data processing method and apparatus, which extracts a first intermediate image block larger than an image block used for correlation calculation from a first image, performs data analysis calculation on the extracted first intermediate image block, stores an intermediate calculation result, and uses the intermediate calculation result in correlation calculation of a plurality of first image blocks included in the first intermediate image block, thereby implementing multiplexing of calculation resources and improving resource utilization.
According to one aspect of the present invention, there is provided an image data processing method applied to a first electronic device, including: extracting a first intermediate image block from a first image, wherein the first image comprises M rows and N columns of pixels, the first intermediate image block comprises I rows and J columns of pixels, M is not less than I, and N is not less than J; calculating the sum of pixel values of each row of pixels in the first intermediate image block, and storing I first values; sequentially calculating the average value of the pixel values of each of (I-m + 1) first image blocks in the first intermediate image blocks by using the stored I first values, wherein the first image blocks comprise m rows and J columns of pixels, and I > m.
According to an embodiment of the present invention, the image data processing method further includes: extracting a second intermediate image block from a second image, wherein the second image has the same size as the first image, and the first intermediate image block and the second intermediate image block have the same size; calculating the sum of pixel values of each row of pixels in the second intermediate image block, and storing I second values; and sequentially calculating an average value of pixel values of each of (I-m + 1) second image blocks in the second intermediate image block using the stored I second values, wherein the first image block and the second image block have the same size.
According to an embodiment of the present invention, the image data processing method further includes: calculating pixel value products of corresponding pixels in the first intermediate image block and the second intermediate image block to obtain I multiplied by J products; calculating the sum of products of each row of products in the I multiplied by J products, and storing I third values; and sequentially calculating the average value of the pixel value products of each of (I-m + 1) first image blocks in the first intermediate image blocks and the corresponding second image block in the second intermediate image block by using the stored I third values.
According to an embodiment of the present invention, the image data processing method further includes: for each of the (I-m + 1) of the first intermediate image blocks, calculating a correlation coefficient between it and a corresponding one of the second intermediate image blocks according to the following formula:
where Ai is the ith first value and represents the sum of the ith row of pixel values in the first intermediate image block, bi is the ith second value and represents the sum of the ith row of pixel values in the second intermediate image block, MABi is the ith third value and represents the sum of the products of the ith row of products in the I × J products, aij represents the ith row and jth column of pixel values in the first intermediate image block, bij represents the ith row and jth column of pixel values in the second intermediate image block, where l is the number of rows in the first intermediate image block in the first row of the first image block.
According to another aspect of the present invention, there is provided an image data processing apparatus applied to a first electronic device, comprising: the image block extraction device comprises an intermediate image block extraction component, a first image extraction component and a second image extraction component, wherein the intermediate image block extraction component is used for extracting a first intermediate image block from a first image, the first image comprises M rows and N columns of pixels, the first intermediate image block comprises I rows and J columns of pixels, M is not less than I, and N is not less than J; a pixel value summing section for calculating a sum of pixel values of pixels of each row in the first intermediate image block and storing I first values; an image block averaging section configured to sequentially calculate an average value of pixel values of each of (I-m + 1) first image blocks in the first intermediate image block using the stored I first values, wherein the first image block includes m rows and J columns of pixels, and wherein I > m.
According to an embodiment of the present invention, in the image data processing apparatus, the intermediate image block extraction means is further configured to extract a second intermediate image block in a second image, the second image having the same size as the first image, the first intermediate image block and the second intermediate image block having the same size; the pixel value summation unit is further configured to calculate a sum of pixel values of each row of pixels in the second intermediate image block and store I second values; the image block averaging part is further configured to sequentially calculate an average value of pixel values of each of (I-m + 1) second image blocks in the second intermediate image block using the stored I second values, where the first image block and the second image block have the same size.
According to an embodiment of the present invention, the image data processing apparatus further includes: a pixel dot multiplication unit configured to calculate a product of pixel values of corresponding pixels in the first intermediate image block and the second intermediate image block to obtain I × J products; dot product summing means for calculating a sum of products of each row of products of the I × J products and storing I third values; and an image block point product averaging unit configured to sequentially calculate, by using the stored I third values, a pixel value product average value of each of (I-m + 1) first image blocks in the first intermediate image block and a corresponding second image block in the second intermediate image block.
According to an embodiment of the present invention, the image data processing apparatus further includes: a correlation coefficient calculating section that calculates, for each of (I-m + 1) of the first intermediate image blocks, a correlation coefficient between it and a corresponding one of the second intermediate image blocks in accordance with the following formula:
where Ai is the ith first value and represents the sum of the ith row of pixel values in the first intermediate image block, bi is the ith second value and represents the sum of the ith row of pixel values in the second intermediate image block, MABi is the ith third value and represents the sum of the products of the ith row of products in the I × J products, aij represents the ith row and jth column of pixel values in the first intermediate image block, bij represents the ith row and jth column of pixel values in the second intermediate image block, where l is the number of rows in the first intermediate image block in the first row of the first image block.
According to an embodiment of the present invention, the first intermediate image block and the second intermediate image block are located in the same row in the first image and the second image, respectively, and the first image block and the second image block are located in the same row in the first image and the second image, respectively.
By adopting the image data processing method and the device according to the embodiment of the invention, the first intermediate image block and the second intermediate image block which are larger than the image block used for correlation calculation are respectively extracted from the first image and the second image, the extracted first intermediate image block and the extracted second intermediate image block are subjected to data analysis calculation and intermediate calculation results are stored, and then the intermediate calculation results are used for correlation calculation of a plurality of first image blocks included in the first intermediate image block and a plurality of second image blocks included in the second intermediate image block, so that multiplexing of calculation resources is realized, and the resource utilization rate is improved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
fig. 1 shows a first image block in a first image and a second image block in a second image for correlation calculation, wherein the first image block and the second image block are both 9 × 9 pixel image blocks;
FIG. 2 illustrates an intermediate image block according to the present inventive concept;
FIGS. 3A, 3B and 3C show schematic flow diagrams of image data processing methods according to embodiments of the invention; and
fig. 4 shows a schematic block diagram of an image data processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, various embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a few embodiments of the invention and are not necessarily all embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments described in the present application without inventive step, shall fall within the scope of protection of the present application.
Here, it is to be noted that, in the drawings, the same reference numerals are given to constituent parts having substantially the same or similar structures and functions, and repeated description thereof will be omitted.
As shown in fig. 1, the first image block and the second image block are both 9 × 9 pixel image blocks, in the conventional image data processing method, when calculating the correlation between the first image block and the second image block, data of the first image block and the second image block are separately calculated, and then, when calculating the correlation between the first image block and the next second image block, intermediate results calculated previously with respect to the second image block are discarded regardless of whether there is an overlap between the second image block and the next second image block. This image data processing method is particularly wasteful of resources in the case where the second image block and the next second image block differ by only one row in the up-down direction and are located in the same column.
The invention provides a new image data processing method, which improves the resource utilization rate by multiplexing the intermediate result of image processing and further improves the speed of image data processing.
As shown in fig. 2, according to the inventive concept, a concept of intermediate image blocks is proposed, which are larger than the image blocks used for correlation calculation, i.e. a first image block in a first image, a second image block in a second image and a next second image block. Specifically, the number of pixel columns included in the intermediate image block is the same as the number of pixel columns included in the image block for correlation calculation, and the number of pixel rows included in the intermediate image block is greater than the number of pixel rows included in the image block for correlation calculation. For example, as shown in fig. 2, the intermediate image block may be an 18 × 9 pixel image block, i.e. it comprises 18 rows and 9 columns of pixels.
According to the concept of the invention, the data of each row of the first intermediate image block in the first image is processed in advance, the data of each row of the second intermediate image block in the second image is processed, the data of each row of the first intermediate image block and the data of the relevant row of the second intermediate image block are combined and processed, the intermediate processing results are stored, and then the intermediate processing results are utilized for multiple times in the correlation calculation between the first image block in the first intermediate image block and the second image block in the second intermediate image block, so that the resource multiplexing is realized, the resource utilization rate is improved, and the image data processing speed is further improved.
As shown in fig. 3A, 3B and 3C, a schematic flow diagram of an image data processing method 300 according to an embodiment of the present invention is shown. The image data processing method 300 according to an embodiment of the present invention is applied to a first electronic device that can perform image processing such as image encoding, image decoding, moving image analysis, and the like. The image processing method 300 according to an embodiment of the present invention is described below taking as an example the correlation calculation between the first image and the second image.
In step S310, a first intermediate image block is extracted from a first image, where the first image includes M rows and N columns of pixels, and the first intermediate image block includes I rows and J columns of pixels, where M is greater than or equal to I, and N is greater than or equal to J.
In step S320, the sum of the pixel values of each row of pixels in the first intermediate image block is calculated, and I first values Ai are stored, wherein I ≧ I ≧ 1.
In step S330, using the stored I first values, sequentially calculating an average value of pixel values of each of (I-m + 1) first image blocks in the first intermediate image block, where the first image block includes m rows and J columns of pixels, where I > m.
In the embodiment of the invention, M, I, J, M and N are all natural numbers.
Thus, in steps S310 to S330, the data of each row of the first intermediate image block is processed to obtain the sum of the pixel values of the pixels of each row of the first intermediate image block (accordingly, the average value of the pixel values of the pixels of each row may also be obtained), and the average value of the pixel values of each of the (I-m + 1) first image blocks included in the first intermediate image block.
Taking the 18 × 9 pixels in fig. 2 as an example, through steps S310-S330, the sum of pixel values (i.e., 18 first values) of each row of pixels in the 18 rows of pixels of the first intermediate image block may be obtained, and the average of pixel values of each of the 10 first image blocks included in the first intermediate image block may be obtained, specifically, the first image block includes the 1 st row to 9 th row pixels of the first intermediate image block, the second first image block includes the 2 nd row to 10 th row pixels of the first intermediate image block, and so on, the tenth first image block includes the 10 th row pixels to 18 th row pixels of the first intermediate image block.
In the correlation calculation with respect to 10 first image blocks and their corresponding second image blocks, the intermediate calculation results as described above may be reused, thereby improving resource utilization.
Further, the image data processing method 300 according to the embodiment of the present invention may also perform a similar operation on the second image.
In step S340, a second intermediate image block is extracted from a second image, where the second image has the same size as the first image, and the first intermediate image block and the second intermediate image block have the same size.
In step S350, the sum of the pixel values of each row of pixels in the second intermediate image block is calculated, and I second values Bi are stored, wherein I is larger than or equal to I and larger than or equal to 1.
In step S360, using the stored I second values, sequentially calculating an average value of pixel values of each of (I-m + 1) second image blocks in the second intermediate image block, wherein the first image block and the second image block have the same size. More advantageously, the first image block has the same position in the first intermediate image block as the second image block has in the second intermediate image block.
Thus, the data of each row of the second intermediate image block is processed in steps S340 to S360, and the sum of the pixel values of the pixels of each row of the second intermediate image block (and accordingly, the average of the pixel values of the pixels of each row may be obtained), and the average of the pixel values of each of (I-m + 1) second image blocks included in the second intermediate image block are obtained.
Continuing with the example of 18 × 9 pixels in fig. 2, through steps S340-S360, the sum of pixel values (i.e., 18 second values) of each of the 18 rows of pixels of the second intermediate image block may be obtained, and the average of pixel values of each of the 10 second image blocks included in the second intermediate image block may be obtained.
In the correlation calculation with respect to 10 first image blocks and their corresponding second image blocks, the intermediate calculation results as described above may be reused, thereby further improving resource utilization.
Furthermore, the image data processing method 300 according to an embodiment of the present invention may also perform a combined processing of data of each row of the first intermediate image block and an associated row of the second intermediate image block.
In step S370, the pixel value products of the corresponding pixels in the first intermediate image block and the second intermediate image block are calculated to obtain I × J products, the sum of the products of each row of the I × J products is calculated, and I third values MABi are stored, wherein I ≧ I ≧ 1
In step S380, using the stored I third values, the average of the pixel value products of each of the (I-m + 1) first image blocks in the first intermediate image block and the corresponding second image block in the second intermediate image block is sequentially calculated.
Continuing with the example of 18 × 9 pixels in fig. 2, through steps S370-S380, 18 third values may be obtained, and 10 pixel-value product averages may be obtained, the 1 st pixel-value product average corresponding to the first image block and the first second image block, the 2 nd pixel-value product average corresponding to the second first image block and the second image block, and so on, the 10 th pixel-value product average corresponding to the tenth first image block and the tenth second image block.
For the application of the correlation calculation between the first image and the second image, in step S390, according to the image processing method 300 of the embodiment of the present invention, for each of the (I-m + 1) first image blocks in the first intermediate image block, the correlation coefficient between the (I-m + 1) first image block and the corresponding one of the second intermediate image blocks is calculated according to the following formula:
where Ai is the ith first value and represents the sum of the ith row of pixel values in the first intermediate image block, bi is the ith second value and represents the sum of the ith row of pixel values in the second intermediate image block, MABi is the ith third value and represents the sum of the products of the ith row of products in the I × J products, aij represents the ith row and jth column of pixel values in the first intermediate image block, bij represents the ith row and jth column of pixel values in the second intermediate image block, where l is the number of rows in the first intermediate image block in the first row of the first image block.
Specifically, I Ai are calculated in step S320, and the average of pixel values of (I-m + 1) first image blocks is calculated in step S330I Bi are calculated in step S350, and the average of pixel values of (I-m + 1) first image blocks is calculated in step S360In step S370, I MABi are calculated, and in step S380, the pixel value product average of (I-m + 1) image blocks is calculated
Advantages of the image data processing method according to the embodiment of the present invention will be specifically exemplified below. In the following example, the original 9 × 9 pixel image block is taken as the basis of comparison.
According to the existing correlation calculation method, specifically referring to formula (1) in the background art section, it is necessary to calculate a pixel value average value of the first image block (including 9 × 9-1=80 adding operations and 1 dividing operation), a pixel value average value of the second image block (including 9 × 9-1=80 adding operations and 1 dividing operation), a pixel value product average value of corresponding pixels of the first image block and the second image block (including 9 × 9=81 multiplying operations, 80 adding operations, and 1 dividing operation), and further, a variance of pixel values of the first image block, a variance of pixel values of the second image block, a product of a variance of pixel values of the first image block and a variance of pixel values of the second image block, and a product of a pixel value average value of the first image block and a pixel value average value of pixel values of the second image block. Since in the image data processing method according to the embodiment of the present invention, the variance of the pixel values of the first image block, the variance of the pixel values of the second image block, the product of the variance of the pixel values of the first image block and the variance of the pixel values of the second image block, and the product of the average of the pixel values of the first image block and the average of the pixel values of the second image block also need to be calculated, the comparison of these parts will be omitted in the following comparison.
Then, the first image block and the second image block are respectively translated downward by one row, and then the correlation between the translated first image block and the translated second image block is calculated, and it is still necessary to calculate the average value of the pixel values of the first image block (including 80 addition operations and 1 division operation), the average value of the pixel values of the second image block (including 80 addition operations and 1 division operation), and the average value of the product of the pixel values of the corresponding pixels of the first image block and the second image block (including 9 × 9=81 multiplication operations, 80 addition operations, and 1 division operation).
That is to say, according to the existing correlation calculation method, for x adjacent first image blocks, in the process of calculating the correlation between each first image block and the corresponding second image block, it is necessary to perform: separate operations for the first image: 80 × x addition operations and 1 × x = x division operations; operation separate for the second image: 80 × x addition operations and 1 × x = x division operations; for a combining operation of the first image and the second image: 81 × x multiplication operations, 80 × x addition operations, and 1 × x division operations.
First example
In this first example, the intermediate image block is taken to comprise 10 x 9 pixels, i.e. the intermediate image block comprises 10 rows and 9 columns of pixels. The first image block and the second image block each comprise 9 x 9 pixels.
According to an embodiment of the present invention, in step S320, 10 × 8=80 addition operations are performed, in step S330, 8 × 2=16 addition operations and 2 division operations are performed, in step S350, 10 × 8=80 addition operations are performed, in step S360, 8 × 2=16 addition operations and 2 division operations are performed, then in step S370, 10 × 9=90 multiplication operations and 10 × 8=80 addition operations are performed, and in step S380, 8 × 2=16 addition operations and 2 division operations are performed.
In other words, in this first example, it is necessary to perform: operations separate for the first image: 96 addition operations and 2 division operations; operation separate for the second image: 96 addition operations and 2 division operations; for a combining operation of the first image and the second image: 90 multiply operations, 96 add operations, and 2 divide operations.
In this first example, only 16 addition operations and 1 division operation are added on the first image side, only 16 addition operations and 1 division operation are added on the second image side, and only 9 multiplication operations, 16 addition operations and 1 division operation are added for the combination operation of the first image and the second image, as compared with the operation of one first image block and one second image block.
However, according to the existing correlation calculation method, for 2 adjacent first image blocks and 2 adjacent second image blocks, it is necessary to perform: operations separate for the first image: 80 × 2 addition operations and 2 division operations; separate operations for the second image: 80 × 2 addition operations and 2 division operations; for a combining operation of the first image and the second image: 81 × 2 multiply operations, 80 × 2 add operations, and 2 divide operations.
Therefore, in this first example, the image data processing method according to the embodiment of the present invention can reduce 192 times of addition operations and 72 times of multiplication operations for 2 adjacent first image blocks and 2 adjacent second image blocks compared to the existing correlation calculation method. Obviously, because part of the intermediate data is multiplexed, the image data processing method according to the embodiment of the invention can improve the resource utilization efficiency and improve the speed of image data processing.
Second example
In this second example, the intermediate image block is taken to comprise 12 x 9 pixels, i.e. the intermediate image block comprises 12 rows and 9 columns of pixels. The first image block and the second image block each comprise 9 x 9 pixels.
According to the embodiment of the present invention, in step S320, 12 × 8=96 addition operations are performed, in step S330, 8 × 4=32 addition operations and 4 division operations are performed, in step S350, 12 × 8=96 addition operations are performed, in step S360, 8 × 4=32 addition operations and 4 division operations are performed, then in step S370, 12 × 9=108 multiplication operations and 12 × 8=96 addition operations are performed, and in step S380, 8 × 4=32 addition operations and 4 division operations are performed.
In other words, in this second example, it is necessary to perform: operations separate for the first image: 128 addition operations and 4 division operations; separate operations for the second image: 128 add operations and 4 divide operations; for a combining operation of the first image and the second image: 108 multiplication operations, 128 addition operations, and 4 division operations.
However, according to the existing correlation calculation method, for 4 sequentially adjacent first image blocks and 4 sequentially adjacent second image blocks, it is necessary to perform: separate operations for the first image: 80 × 4 addition operations and 4 division operations; operation separate for the second image: 80 × 4 addition operations and 4 division operations; for a combining operation of the first image and the second image: 81 × 4 multiplication operations, 80 × 4 addition operations, and 4 division operations.
Therefore, in this second example, the image data processing method according to the embodiment of the present invention can reduce 576 addition operations and 216 multiplication operations for 4 sequentially adjacent first image blocks and 4 sequentially adjacent second image blocks, as compared to the existing correlation calculation method. Obviously, as more intermediate data are multiplexed, the image data processing method according to the embodiment of the present invention can improve the resource utilization efficiency and increase the speed of image data processing.
Third example
In this second example, the intermediate image block is taken to comprise 15 x 9 pixels, i.e. the intermediate image block comprises 15 rows and 9 columns of pixels. The first image block and the second image block each comprise 9 x 9 pixels.
According to the embodiment of the present invention, in step S320, 15 × 8=120 addition operations are performed, in step S330, 8 × 7=56 addition operations and 7 division operations are performed, in step S350, 15 × 8=120 addition operations are performed, in step S360, 8 × 7=56 addition operations and 7 division operations are performed, then in step S370, 15 × 9=135 multiplication operations and 15 × 8=120 addition operations are performed, and in step S380, 8 × 7=56 addition operations and 7 division operations are performed.
In other words, in this second example, it is necessary to perform: separate operations for the first image: 176 addition operations and 7 division operations; separate operations for the second image: 176 addition operations and 7 division operations; for a combining operation of the first image and the second image: 135 multiply operations, 176 add operations, and 7 divide operations.
However, according to the existing correlation calculation method, for 7 sequentially adjacent first image blocks and 7 sequentially adjacent second image blocks, it is necessary to perform: separate operations for the first image: 80 × 7 addition operations and 7 division operations; separate operations for the second image: 80 × 7 addition operations and 7 division operations; for a combining operation of the first image and the second image: 81 × 7 multiply operations, 80 × 7 add operations, and 7 divide operations.
Therefore, in this third example, the image data processing method according to the embodiment of the present invention can reduce 1152 addition operations and 432 multiplication operations for 4 adjacent first image blocks and 4 adjacent second image blocks, compared to the existing correlation calculation method. Obviously, the image data processing method according to the embodiment of the present invention can improve the resource utilization efficiency and increase the speed of image data processing because more intermediate data are multiplexed.
As is apparent from the above example, as the number of pixel rows of the intermediate image block increases, the amount of computation in the computation process is greatly reduced. This, of course, requires more intermediate data to be stored. Thus, there may be a good compromise between the amount of intermediate data storage and the image processing speed, and the size of the intermediate image blocks may be chosen appropriately according to the actual device storage capacity and the actual processing speed requirements. It should be appreciated that embodiments of the present invention are not limited by the size of the intermediate image block, in other words, embodiments of the present invention may cover any suitable intermediate image block as long as the number of pixel columns of the intermediate image block is the same as the number of pixel columns of the image block for correlation calculation and the number of pixel rows of the intermediate image block is greater than the number of pixel rows of the image block for correlation calculation.
The image data processing method according to an embodiment of the present invention is particularly suitable for such a limit case where the first image is translated in the horizontal direction or the vertical direction to obtain the second image.
In this case, the first intermediate image block extracted in the first image and the second intermediate image block extracted in the second image not only have the same size but also may be located on the same row in the first image and the second image, respectively. Furthermore, in this case, the first image block in the first image and the corresponding second image block in the second image not only have the same size, but are also located in the same row in the first image and the second image, respectively.
An image data processing apparatus 400 according to an embodiment of the present invention will be described below with reference to fig. 4, the image data processing apparatus 400 according to an embodiment of the present invention being applied to a first electronic device.
As shown in fig. 4, the image data processing apparatus 400 according to an embodiment of the present invention includes: an intermediate image block extracting part 410, a pixel value summing part 420, and an image block averaging part 430.
The intermediate image block extraction part 410 extracts a first intermediate image block in a first image, the first image comprising M rows and N columns of pixels, the first intermediate image block comprising I rows and J columns of pixels, wherein M is greater than or equal to I, and N is greater than or equal to J.
The pixel value summation component 420 calculates the sum of the pixel values of each row of pixels in the first intermediate image block and stores I first values.
Image block averaging section 430 sequentially calculates an average value of pixel values of each of (I-m + 1) first image blocks in the first intermediate image blocks, which include m rows and J columns of pixels, using the stored I first values, wherein I > m.
After the calculation on the first image side is completed, the image data processing apparatus 400 may perform the calculation on the second image side.
The intermediate image block extraction section 410 also extracts a second intermediate image block in a second image, the second image having the same size as the first image, the first intermediate image block and the second intermediate image block having the same size.
The pixel value summing component 420 further calculates the sum of the pixel values of each row of pixels in the second intermediate image block and stores I second values.
The image block averaging section 430 further sequentially calculates an average value of pixel values of each of (I-m + 1) second image blocks among the second intermediate image blocks, using the stored I second values, wherein the first image block and the second image block have the same size.
Further, the image data processing apparatus 400 may further include: a pixel dot multiplication section 440, a dot multiplication summation section 450, and an image block dot multiplication averaging section 460.
The pixel dot product unit 440 calculates the pixel value products of the corresponding pixels in the first intermediate image block and the second intermediate image block to obtain I × J products.
The dot product summing part 450 calculates the sum of products of each row of the I × J products, and stores I third values.
The image block point multiplication averaging part 460 sequentially calculates the average of the pixel value products of each of the (I-m + 1) first image blocks in the first intermediate image blocks and the corresponding second image block in the second intermediate image block, using the stored I third values.
Further, the image data processing apparatus 400 may further include: the correlation coefficient calculation section 470.
The correlation coefficient calculating section 470 calculates, for each of the (I-m + 1) ones of the first image blocks, a correlation coefficient between it and a corresponding one of the second intermediate image blocks in accordance with equation (2) above.
Advantageously, the first and second intermediate image blocks are located in the same row in the first and second images, respectively, and the first and second image blocks are located in the same row in the first and second images, respectively.
According to the image data processing method and device provided by the embodiment of the invention, the first intermediate image block and the second intermediate image block which are larger than the image blocks used for correlation calculation are respectively extracted from the first image and the second image, the extracted first intermediate image block and the extracted second intermediate image block are subjected to data analysis calculation, the intermediate calculation result is stored, and then the intermediate calculation result is used for correlation calculation of a plurality of first image blocks included in the first intermediate image block and a plurality of second image blocks included in the second intermediate image block, so that multiplexing of calculation resources is realized, and the resource utilization rate is improved.
According to the image data processing method and device provided by the embodiment of the invention, the multiplexing of the computing resources can be realized by realizing favorable compromise between the storage capacity of the storage unit and the computing resource consumption, so that the utilization rate of the computing resources is improved.
Through the above description of the embodiments, those skilled in the art will clearly understand that the present invention may be implemented by means of software plus a necessary hardware platform, and may also be implemented by software or hardware entirely. With this understanding in mind, all or part of the technical solutions of the present invention that contribute to the background can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods according to the embodiments or some parts of the embodiments.
Various embodiments of the present invention are described in detail above. However, those skilled in the art will appreciate that various modifications, combinations, or sub-combinations of the embodiments may be made without departing from the spirit and principle of the invention, and such modifications are intended to be within the scope of the invention.
Claims (4)
1. An image data processing method is applied to a first electronic device and comprises the following steps:
extracting a first intermediate image block from a first image, wherein the first image comprises M rows and N columns of pixels, the first intermediate image block comprises I rows and J columns of pixels, M is not less than I, and N is not less than J;
calculating the sum of pixel values of each row of pixels in the first intermediate image block, and storing I first values;
sequentially calculating the pixel value average value of each of (I-m + 1) first image blocks in the first intermediate image block by using the stored I first values, wherein the first image blocks comprise m rows and J columns of pixels, and I > m;
extracting a second intermediate image block in a second image, wherein the second image has the same size as the first image, and the first intermediate image block and the second intermediate image block have the same size;
calculating the sum of pixel values of each row of pixels in the second intermediate image block, and storing I second values;
sequentially calculating an average value of pixel values of each of (I-m + 1) second image blocks in the second intermediate image block using the stored I second values, wherein the first image block and the second image block have the same size; calculating pixel value products of corresponding pixels in the first intermediate image block and the second intermediate image block to obtain I multiplied by J products;
calculating the sum of products of each row of products in the I multiplied by J products, and storing I third values; and
sequentially calculating the pixel value product average value of each first image block in (I-m + 1) first image blocks in the first intermediate image blocks and the corresponding second image block in the second intermediate image blocks by using the stored I third values;
for each of the (I-m + 1) of the first intermediate image blocks, calculating a correlation coefficient between it and a corresponding one of the second intermediate image blocks according to the following formula:
where Ai is the ith first value and represents the sum of the ith row of pixel values in said first intermediate image block, bi is the ith second value and represents the sum of the ith row of pixel values in said second intermediate image block, MABi is the ith third value and represents the sum of the products of the ith row of products in said I × J products, aij represents the ith row and jth column of pixel values in the first intermediate image block, bij represents the ith row and jth column of pixel values in the second intermediate image block, where l is the number of rows in said first intermediate image block for the first row of said first image block.
2. The image data processing method according to claim 1, wherein the first intermediate image block and the second intermediate image block are located in the same row in the first image and the second image, respectively,
the first image block and the second image block are respectively located in the same row in the first image and the second image.
3. An image data processing device applied to a first electronic device comprises:
the image block extraction device comprises an intermediate image block extraction component, a first image extraction component and a second image extraction component, wherein the intermediate image block extraction component is used for extracting a first intermediate image block from a first image, the first image comprises M rows and N columns of pixels, the first intermediate image block comprises I rows and J columns of pixels, M is not less than I, and N is not less than J;
a pixel value summing section for calculating a sum of pixel values of pixels of each row in the first intermediate image block and storing I first values;
an image block averaging section configured to sequentially calculate an average value of pixel values of each of (I-m + 1) first image blocks in the first intermediate image block, using the stored I first values, where the first image block includes m rows and J columns of pixels, where I > m; the intermediate image block extraction means is further configured to extract a second intermediate image block in a second image, the second image having the same size as the first image, the first intermediate image block and the second intermediate image block having the same size;
the pixel value summation unit is further configured to calculate a sum of pixel values of each row of pixels in the second intermediate image block and store I second values;
the image block averaging part is further configured to sequentially calculate an average value of pixel values of each of (I-m + 1) second image blocks in the second intermediate image block using the stored I second values, where the first image block and the second image block have the same size;
a pixel dot multiplication unit configured to calculate a product of pixel values of corresponding pixels in the first intermediate image block and the second intermediate image block to obtain I × J products;
dot product summing means for calculating a sum of products of each row of products of the I × J products and storing I third values; and
an image block point multiplication and averaging unit configured to sequentially calculate, by using the stored I third values, an average value of pixel value multiplication products of each of (I-m + 1) first image blocks in the first intermediate image block and a corresponding second image block in the second intermediate image block; a correlation coefficient calculating section that calculates, for each of (I-m + 1) of the first intermediate image blocks, a correlation coefficient between it and a corresponding one of the second intermediate image blocks in accordance with the following formula:
where Ai is the ith first value and represents the sum of the ith row of pixel values in said first intermediate image block, bi is the ith second value and represents the sum of the ith row of pixel values in said second intermediate image block, MABi is the ith third value and represents the sum of the products of the ith row of products in said I × J products, aij represents the ith row and jth column of pixel values in the first intermediate image block, bij represents the ith row and jth column of pixel values in the second intermediate image block, where l is the number of rows in said first intermediate image block for the first row of said first image block.
4. The image data processing device of claim 3, wherein the first intermediate image block and the second intermediate image block are located in the same row in a first image and a second image, respectively,
the first image block and the second image block are respectively located in the same row in the first image and the second image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410253811.2A CN105335987B (en) | 2014-06-09 | 2014-06-09 | Image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410253811.2A CN105335987B (en) | 2014-06-09 | 2014-06-09 | Image processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105335987A CN105335987A (en) | 2016-02-17 |
| CN105335987B true CN105335987B (en) | 2018-03-27 |
Family
ID=55286492
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410253811.2A Active CN105335987B (en) | 2014-06-09 | 2014-06-09 | Image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105335987B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106296614B (en) * | 2016-08-17 | 2019-12-13 | 北京集创北方科技股份有限公司 | Image processing apparatus and image processing method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6285711B1 (en) * | 1998-05-20 | 2001-09-04 | Sharp Laboratories Of America, Inc. | Block matching-based method for estimating motion fields and global affine motion parameters in digital video sequences |
| CN101290747A (en) * | 2008-06-18 | 2008-10-22 | 北京工业大学 | Image displaying method of electronic paper screen |
| CN103150710A (en) * | 2013-03-25 | 2013-06-12 | 郑州轻工业学院 | Rapid mean filtering method for image |
| CN103295022A (en) * | 2012-02-24 | 2013-09-11 | 富泰华工业（深圳）有限公司 | Image similarity calculation system and method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4605931B2 (en) * | 2001-03-30 | 2011-01-05 | 株式会社リコー | Image processing apparatus, image processing method, program for causing computer to execute image processing method, and computer-readable recording medium recording the program |
-
2014
- 2014-06-09 CN CN201410253811.2A patent/CN105335987B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6285711B1 (en) * | 1998-05-20 | 2001-09-04 | Sharp Laboratories Of America, Inc. | Block matching-based method for estimating motion fields and global affine motion parameters in digital video sequences |
| CN101290747A (en) * | 2008-06-18 | 2008-10-22 | 北京工业大学 | Image displaying method of electronic paper screen |
| CN103295022A (en) * | 2012-02-24 | 2013-09-11 | 富泰华工业（深圳）有限公司 | Image similarity calculation system and method |
| CN103150710A (en) * | 2013-03-25 | 2013-06-12 | 郑州轻工业学院 | Rapid mean filtering method for image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105335987A (en) | 2016-02-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102325602B1 (en) | Device and method to process data in parallel | |
| JP2022118201A (en) | Image processing system, image processing method, and program | |
| CN109960980B (en) | Dynamic gesture recognition method and device | |
| KR101677559B1 (en) | Image registration device and operation method thereof | |
| KR101107254B1 (en) | Method for estimating motion vector using motion vector of near block and apparatus therefor | |
| CN108961316B (en) | Image processing method and device and server | |
| CN110246095B (en) | Image restoration method and device, storage medium and terminal equipment | |
| CN110874636A (en) | Neural network model compression method and device and computer equipment | |
| CN106780336A (en) | A kind of image downscaling method and device | |
| Zhang et al. | PFFN: Progressive feature fusion network for lightweight image super-resolution | |
| CN105335987B (en) | Image processing method and device | |
| CN107203764A (en) | Long microblog picture recognition methods and device | |
| CN110782398B (en) | Image processing method, generative countermeasure network system and electronic device | |
| CN114049491A (en) | Fingerprint segmentation model training method, fingerprint segmentation device, fingerprint segmentation equipment and fingerprint segmentation medium | |
| CN107644393B (en) | GPU-based abundance estimation algorithm parallel implementation method | |
| JP2014014064A5 (en) | ||
| Rodríguez et al. | Video background modeling under impulse noise | |
| CN111079904B (en) | Acceleration method of depth separable convolution and storage medium | |
| CN104902260B (en) | The acquisition methods and system of a kind of image parallactic | |
| KR101418524B1 (en) | Hardware apparatus and method for generating integral image thereof | |
| CN103927740B (en) | The method and apparatus for obtaining the point spread function of image | |
| CN112712461A (en) | Image deconvolution processing method and device and terminal equipment | |
| CN108765382B (en) | Abundance estimation parallel computing method based on GPU | |
| CN115063299B (en) | Image preprocessing method and device, electronic equipment and storage medium | |
| US20230205926A1 (en) | System and method for preserving privacy for a set of data packets |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |