CN110537202A - Correlation arithmetic unit - Google Patents
Correlation arithmetic unit Download PDFInfo
- Publication number
- CN110537202A CN110537202A CN201780089319.8A CN201780089319A CN110537202A CN 110537202 A CN110537202 A CN 110537202A CN 201780089319 A CN201780089319 A CN 201780089319A CN 110537202 A CN110537202 A CN 110537202A
- Authority
- CN
- China
- Prior art keywords
- pixel
- value
- reference block
- correlation
- arithmetic unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000009826 distribution Methods 0.000 claims abstract description 72
- 238000004364 calculation method Methods 0.000 claims abstract description 60
- 238000006243 chemical reaction Methods 0.000 claims abstract description 29
- 230000009467 reduction Effects 0.000 claims abstract description 22
- 239000004744 fabric Substances 0.000 claims 1
- 238000010977 unit operation Methods 0.000 abstract description 2
- 238000003860 storage Methods 0.000 description 17
- 238000013500 data storage Methods 0.000 description 11
- 238000001514 detection method Methods 0.000 description 5
- 238000011946 reduction process Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 235000005121 Sorbus torminalis Nutrition 0.000 description 2
- 244000152100 Sorbus torminalis Species 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000012886 linear function Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000000034 method Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/32—Determination of transform parameters for the alignment of images, i.e. image registration using correlation-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/223—Analysis of motion using block-matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Reference block in correlation arithmetic unit operation image, the correlation between at least one reference block in described image or in different images, in the correlation arithmetic unit, include pixel Distribution value calculation part, its pixel Distribution value for calculating the pixel in described image, and the range for the pixel value for selecting pixel number more, as valid pixel value range；Conversion coefficient calculation part calculates the conversion coefficient that the pixel of the valid pixel value range is converted to defined gray scale；And pixel value converter section carries out gray scale reduction processing to the pixel in the pixel and the reference block in the reference block according to the conversion coefficient.
Description
Technical field
The present invention relates to the correlation arithmetic units that operation is carried out between the correlation image.
Background technique
In noise reduction process or motion vector detection in image procossing etc., the correlation operation between image is often carried out.
Correlation indicates the similarity between image.Correlation between 2 images (side being referred to as reference frame, another party is referred to as reference frame)
Specific region (reference block) in reference frame, with the specific region in the reference frame with block size identical with reference block
Between (reference block), calculated according to the difference etc. of the mutual pixel value of the pixel of same position.Calculating for correlation,
It has been known that there is difference absolute values and the calculation methods such as (SAD), difference quadratic sum (SSD).
In noise reduction process, motion vector detection etc., needs to be carried out at high speed the correlation and calculate fortune.Therefore, noise reduction is being carried out
In the device of processing, motion vector detection etc., setting carries out the special circuit (correlation arithmetic unit) of correlation operation mostly.
With the high resolution of the image as correlation operand, the bit number of each pixel unit increases, from
And the circuit scale of correlation arithmetic unit increases.It is required that in the feelings for the correlation arithmetic speed for not reducing correlation arithmetic unit
Under condition, the circuit scale of increase is cut down.
Following position alignment device is described in patent document 1: grey using carrying out according to the pixel Distribution value of image
It spends low-resolution image obtained by reduction processing and carries out matching treatment, which reduces the bit that processing reduces each pixel unit
Number.Position alignment device can cut down matching treatment by using low-resolution image obtained by gray scale reduction processing is carried out
Circuit scale, and matching treatment can be implemented at a high speed.
Existing technical literature
Patent document
Patent document 1: Japanese Unexamined Patent Publication 10-222669 bulletin
Summary of the invention
Problems to be solved by the invention
In the position alignment device documented by patent document 1, maximum value for pixel value is generated to minimum value
The pixel of entire pixel value range carries out low-resolution image obtained by gray scale reduction processing.
But the correlation operation carried out in the device for carrying out noise reduction process, motion vector detection etc. is according to image district
Characteristics of image in domain finds out the similarity between image.
Therefore, in correlation operation, when using low-resolution image obtained by gray scale reduction processing is carried out, it is expected that with
The mode of pixel value distribution trend is maintained to generate low-resolution image as far as possible.
In view of the foregoing, the purpose of the present invention is to provide a kind of correlation arithmetic units, by being able to maintain that image
The gray scale reduction of feature is handled, and in the case where not reducing correlation arithmetic speed, can cut down circuit scale.
Means for solving the problems
In order to solve the above problems, the invention proposes following means.
Reference block in correlation arithmetic unit operation image of the invention, in described image or in other images extremely
Correlation between a few reference block, in the correlation arithmetic unit, comprising: pixel Distribution value calculation part calculates
The pixel Distribution value of pixel in described image, and the range for the pixel value for selecting pixel number more and as valid pixel value model
It encloses；Conversion coefficient calculation part calculates the conversion coefficient that the pixel of the valid pixel value range is converted to defined gray scale；
And pixel value converter section, according to the conversion coefficient, to the pixel in the pixel and the reference block in the reference block
Carry out gray scale reduction processing.
The effect of invention
Correlation arithmetic unit according to the present invention, it is possible to provide a kind of by being able to maintain that at the gray scale reduction of characteristics of image
Reason, in the case where not reducing correlation arithmetic speed, cuts down the correlation arithmetic unit of circuit scale.
Detailed description of the invention
Fig. 1 is the figure for showing the correlation arithmetic unit of one embodiment of the present invention.
Fig. 2 is the figure for showing the relationship between reference block and reference block.
Fig. 3 is the gray scale for showing the pixel value converter section in the correlation arithmetic unit of one embodiment of the present invention and carrying out
The figure of data flow in reduction processing.
Fig. 4 is that the pixel Distribution value calculation part shown in the correlation arithmetic unit of one embodiment of the present invention calculates
Histogram computing object region figure.
Fig. 5 is that the pixel Distribution value calculation part shown in the correlation arithmetic unit of one embodiment of the present invention calculates
Histogram figure.
Fig. 6 is the deformation for showing the pixel Distribution value calculation part in the correlation arithmetic unit of one embodiment of the present invention
The figure of the calculated respective histogram of RGB of example.
Fig. 7 is the deformation for showing the pixel Distribution value calculation part in the correlation arithmetic unit of one embodiment of the present invention
The figure of the calculated histogram of example.
Specific embodiment
(embodiment)
Illustrate an embodiment of correlation arithmetic unit of the invention referring to Figure 1 to Figure 7.
The correlation arithmetic unit 100 of present embodiment is installed in the dress for carrying out noise reduction process, motion vector detection etc.
In setting, with supercomputing correlation.
Fig. 1 is the figure for showing the structure of correlation arithmetic unit 100.Correlation arithmetic unit 100 calculates 1 image (frame)
It is interior specific region (reference block), related between the specific region (reference block) with block size identical with reference block
Value.
As shown in Fig. 2, the correlation arithmetic unit 100 of present embodiment calculate with the concerned pixel of image (frame) (tx,
Ty) the correlation between the reference block and at least one reference block of center pixel.
In explanation later, in order to illustrate the position of the pixel of image, XY coordinate is used.As shown in Fig. 2, setting level side
To be Y direction for X-direction, vertical direction, upper left is origin.XY coordinate such as (X, Y)=(0,0) shows like that.
There is correlation arithmetic unit 100 main memory 1, block storage 2, correlation operational part 3, pixel Distribution value to calculate
Portion 5, conversion coefficient calculation part 6 and pixel value converter section 7.
Main memory 1 is by structures such as DRAM (Dynamic Random Access Memory: dynamic random access memory)
At image data (frame data 10) of the storage as correlation operand.Here, frame data 10 are divided into multiple benchmark
Block.
One side (can also be referred to as reference frame, another party is referred to as reference for 2 images by correlation arithmetic unit 100
Frame), specific region (reference block) in reference frame, with the given zone in the reference frame with block size identical with reference block
Between domain (reference block), high-speed computation correlation.
In the case that correlation arithmetic unit 100 calculates correlation between 2 images (reference frame and reference frame), in master
Memory Reference frame data 11 and the both sides referring to frame data 12 in memory 1.
Block storage 2 is memory of the storage as the data of the reference block and reference block of correlation operand, is had
Reference block data storage memory 21 and reference block data storage memory 22.
The storage of reference block data with memory 21, by SRAM, (Static Random Access Memory: deposit by static random
Access to memory) or register file etc. constitute, in the frame data 10 that storage main memory 1 is stored, at least one reference block
Data.Not shown CPU of the data of reference block etc. is read from main memory 1, and is sent to the storage of reference block data
With memory 21.
Reference block data storage memory 22 is made of SRAM or register file etc., and storage main memory 1 is stored
Frame data 10 in, the data of multiple reference blocks.Not shown CPU of the data of reference block etc. is read from main memory 1,
And it is sent to reference block data storage memory 22.
As shown in Fig. 2, reference point (reference pixels) is using concerned pixel as center pixel, water in frame data 10
Square into the region of 17 pixels of 17 pixels and vertical direction (being referred to as " 17 × 17 " later), from being configured to alternately trellis diagram
144 pixels after concerned pixel are removed in the pixel of case shape (chequer shape).Reference block data storage memory 22 is deposited
The data in region (reference area) of the storage comprising 144 reference blocks, these reference blocks are with reference point (rx, ry) for center pixel.
It here, is reference block itself by the block of center pixel of concerned pixel, therefore the phase between calculating and reference block
It is removed in the object of pass value.
Here, reference point be configured to alternately grid pattern shape (chequer shape) refer to the pixel as reference point, with
It is alternately arranged in the horizontal direction and the vertical direction every a pixel not as the pixel of reference point.
As shown in Fig. 2, the block size that the block size of reference block and reference block is 9 × 9.
The data that reference block and reference block are stored are the data that each pixel is 12 bits.Data can be monochrome
Or any one in color data, it is also possible to the intermediate data from imaging sensor.
Correlation between 3 operation reference block of correlation operational part and reference block.
Fig. 3 is the figure for showing the data flow that the gray scale that aftermentioned pixel value converter section 7 carries out reduces processing.As shown in figure 3,
Pixel data as the data that each pixel is 12 bits, reference block pixel data and reference block is turned by pixel value
It changes portion 7 and carries out gray scale reduction processing, be converted into the pixel data that each pixel is 6 bits.Low resolution figure generated
The data of picture are sent to correlation operational part 3.Correlation operational part 3 according to as each pixel be 6 bits data,
The pixel data of reference block and the pixel data of reference block, operation correlation.
Correlation operational part 3 is the circuit of 144 correlations between 1 reference block of operation and 144 reference blocks.
The reference block stored via aftermentioned 7 input reference block number of pixel value converter section according to storage memory 21
Data and the data of 144 reference blocks that are stored of reference block data storage memory 22.
Correlation operational part 3 is configured to and 144 correlations of column count, is also configured to carry out in a time division manner
It calculates.
3 operation difference absolute value of correlation operational part and (SAD) are used as correlation.SAD in reference point (rx, ry) passes through
Formula 1 below calculates.
In formula 1, image (x, y) is the pixel data of the image (frame data 10) at XY coordinate (x, y).It can be list
Any one in color or color data, is also possible to the intermediate data from imaging sensor.Size is indicated from center pixel
To the pixel number -1 of most end pixel.
The operation of correlation between reference block and reference block can operation blocks entirety together correlation, can also calculate
The correlation of one column of block size, and the correlation cumulative addition that calculated one arranges is come to the correlation of operation block size.
Pixel Distribution value calculation part 5 is the electricity that pixel Distribution value is calculated according to the reference frame as correlation operand
Road.The pixel Distribution value calculation part 5 of present embodiment calculates histogram, as pixel Distribution value.Aftermentioned pixel value converter section 7
According to the calculated histogram of pixel Distribution value calculation part 5, gray scale reduction processing is carried out, is generated as correlation operand
Low-resolution image.
Fig. 4 is the figure for showing the computing object region of histogram of the calculating of pixel Distribution value calculation part 5.(a) institute example of Fig. 4
The image of reference frame is integrally used as the computing object region of histogram by the figure shown.Scheme reference frame illustrated by (b) of Fig. 4 only
Computing object region of the reference block as histogram.
The computing object region of histogram can the image of frame be whole on the basis of, may also be only reference block, but such as Fig. 4
(b) shown in, the pixel Distribution value calculation part 5 of present embodiment is calculated according only to the data of reference block as pixel Distribution value
Histogram.
Characteristics of image in correlation operation image-region local according to as reference block or reference block, finds out part
Image between similarity.
Therefore, in correlation operation, when using low-resolution image obtained by gray scale reduction processing is carried out, preferably not
It is the pixel value distribution trend to maintain the image entirety of reference frame, but to maintain the side of the pixel value distribution trend of part
Formula generates low-resolution image.Therefore, the histogram used to generate low-resolution image is preferably capable grasping part
Pixel value distribution trend histogram.
Therefore, in the present embodiment, the data of the reference block stored according only to block storage calculate and are used as pixel value
The histogram of distribution.
In addition, pixel Distribution value calculation part 5 can also be other than the data according to reference block, also according to the number of reference block
According to calculating histogram.By using as correlation operand, reference block and reference block both sides data, can calculate
The histogram of local pixel value distribution trend can more be grasped.
Pixel Distribution value calculation part 5 reads the data of all 81 pixels for the reference block that block size is 9 × 9, and by every
The gray scale (pixel value) of a 12 bit counts pixel number, thus calculates histogram.Here, the counting of pixel number can also be with
It is not the gray scale of every 12 bit.For example, 4 bit of low level of pixel value can be given up, and press the gray scale (pixel value) of each 8 bit
Pixel number is counted, histogram is thus calculated.In this way, also will appreciate that pixel value distribution trend, and can
Cut down the circuit scale for being used for histogram calculation.
Pixel Distribution value calculation part 5 can also be included according to the calculated result of histogram in conjunction with the calculating of histogram
The determination of the valid pixel value range of characteristic pixel Distribution value.In the present embodiment, pixel Distribution value calculation part 5 is according to meter
The histogram of calculating calculates the upper limit value hist_high and lower limit value hist_low of valid pixel value range.By the upper limit value
Range between hist_high and lower limit value hist_low is set as valid pixel value range.
Fig. 5 shows the calculated histogram of pixel Distribution value calculation part 5 and upper limit value hist_high and lower limit value
hist_low.In the prior art, for the entire pixel of the maximum value (white) of the pixel value in histogram to minimum value (black)
The pixel for being worth range carries out gray scale reduction processing, generates the low-resolution image as correlation operand.
On the other hand, as shown in figure 5, the correlation arithmetic unit 100 of present embodiment is by the pixel value in histogram
The range of upper limit value hist_high to lower limit value hist_low are set as valid pixel value range, and to the valid pixel value range
Pixel carry out gray scale reduction processing, generate low-resolution image as correlation operand.
There is following trend as shown in Figure 5: the pixel value in histogram in the image data of common natural image
Median near, pixel number increases, with the maximum value (white) or minimum value (black) close to the pixel value in histogram, pixel
Number is reduced.The less pixel value range of pixel number is compared with the more pixel value range of pixel number, the characteristic not comprising image
The case where pixel Distribution value, is more.
Therefore, as shown in figure 5, the pixel Distribution value calculation part 5 of present embodiment will be except the maximum value close to pixel value
Range other than the range of (white) and minimum value (black) is set as valid pixel value range, the maximum value (white) close to pixel value and
The range of minimum value (black) is predicted to be the less range of pixel number.
Upper limit value hist_high is set as in whole pixel used in histogram calculation, from the maximum picture of pixel value
Element rises successively remove HIST_HIGH_NUM pixel after max pixel value in remaining pixel.
Lower limit value hist_low is set as in whole pixel used in histogram calculation, from the smallest pixel of pixel value
Play the minimum pixel value after successively removing HIST_LOW_NUM pixel in remaining pixel.
In addition, HIST_HIGH_NUM and HIST_LOW_NUM can change numerical value by software-based setting.
By removing certain pixel number, a part missing of pixel data from pixel value distribution.But it is related
The purpose that value calculates is similarity judge between reference block and reference block, therefore the close mutual difference of pixel of pixel value is believed
Breath is important.Therefore, being distributed in the less pixel value of the maximum value of pixel distribution scope, the pixel number near minimum value is pair
The low pixel of importance for similitude judgement, it is believed that it is lower that bring, which influences, to be judged on similitude.
Here, upper limit value hist_high and lower limit value hist_low is it is also contemplated that the random noise, solid that image is included
The influence of mould-fixed noise selects.
In particular, as shown in figure 5, fixed pattern noise is in the maximum value (white) and minimum value (black) for becoming pixel value
Trend.Therefore, only by the way that certain pixel value range will be removed from the maximum value of pixel value distribution and minimum value after
Range is set as valid pixel value range, it is difficult to the selection of the valid pixel value range of robust is carried out for fixed pattern noise.
As the pixel Distribution value calculation part 5 of present embodiment, by removing a certain number of biggish picture of pixel value
Element and the lesser pixel of pixel value can carry out the selection of the valid pixel value range of robust for fixed pattern noise.
Here, upper limit value hist_high and lower limit value hist_low can be based on (ISO sensitivity, fast according to shooting condition
Door speed, temperature etc.) generation rate of fixed pattern noise of estimation calculated.According to the pixel number and fixed mode of image
The generation number of noise is calculated relative to the pixel number used in histogram calculation, fixed pattern noise estimation pixel number,
And valid pixel value range is set as by the range after the pixel number is removed.
The pixel number of image is being set as PIX_NUM, the generation number of stain/white point fixed pattern noise is being set to
In the case where FIXED_LOW_NUM, FIXED_HIGH_NUM, relative to the pixel number HIST_NUM used in histogram calculation
Stain, white point estimation pixel number HIST_LOW_NUM, HIST_HIGH_NUM of fixed pattern noise pass through formula 2 below
It is calculated with formula 3.
Conversion coefficient calculation part 6 is calculated for by the picture of the selected valid pixel value range of pixel Distribution value calculation part 5
Element is converted to the conversion coefficient of defined gray scale.
Conversion coefficient calculation part 6 calculates the mutually decrement offset and shift amount shift of pixel value, as conversion coefficient.In
In the case where reducing that treated by gray scale gray scale has been set as 6 bits, the mutually decrement offset and shift amount shift of pixel value are such as
It calculates like that below.
Firstly, as illustrated below in formula 4, the mutually decrement offset of pixel value is set as being counted by pixel Distribution value calculation part 5
The lower limit value hist_low of calculating.
Offset=hist_low ... formula 4
Then, as shown in formula 5 below, effective picture is calculated according to upper limit value hist_high and lower limit value hist_low
Element value range range.
Range=hist_higt-hist_low ... formula 5
Finally, as shown in formula 6 below, according to gray scale reduce that treated gray scale (6 bit) and valid pixel value range
Range calculates shift amount shift.
In addition, as shown in formula 7 below, shift amount shift can also be calculated in the case where image data takes negative value
For negative value.
Pixel value converter section 7 is according to the calculated conversion coefficient of conversion coefficient calculation part 6, to the number of reference block and reference block
According to gray scale reduction processing is carried out, the low-resolution image as correlation operand is generated.Low-resolution image generated
Data be sent to correlation operational part 3.
The reference block of correlation is calculated in design and the block size of reference block is BLOCK_SIZE, the pixel value of reference block is
Base_block [i] [j], reference block pixel value be ref_block [i] [j].Here, [i] [j] indicates X-coordinate for i, Y seat
It is designated as the pixel of j.
Firstly, being subtracted from base_block [i] [j] and ref_block [i] [j] by converting as shown in formula 8 below
The calculated offset of coefficient calculation part 6, to calculate base_offset [i] [j] and ref_offset [i] [j].
Then, as shown in formula 9 below, by making base_offset [i] [j] and ref_offset [i] [j] to convert
The calculated shift amount shift right shift (logical shift) of coefficient calculation part 6 calculates base_shift [i] [j] and reference block
Pixel value ref_shift [i] [j].
Finally, as shown in formula 10 below, with the maximum value of 6 bits to base_shift [i] [j] and ref_shift [i]
[j] carries out clipping, calculates the base_out [i] [j] and ref_out [i] [j] of the output as pixel value converter section 7,6 ratio
Spy is that gray scale reduces treated gray scale.
(0≤i, j < BLOCK_SIZE) ... formula 10
Then, the movement of correlation arithmetic unit 100 is illustrated.
Firstly, object frame data 10 are stored in main memory 1.Even if not by the total data storage of frame data 10 to master
Memory 1 can also be stored in the data for the data and reference block that will be used as the reference block of correlation operand to primary storage
At the time of device 1, the correlation operation of object block is successively executed.
The data of reference block as operand arrive reference block data storage memory 21 by successively storage.In addition,
The data of the reference block of reference area corresponding with the reference block as operand are stored in the storage of reference block data with depositing
Reservoir 22.
At the time of the storage of the data of reference block of operand will be used as to reference block data storage memory 21, as
Plain Distribution value calculation part 5 calculates histogram and upper limit value hist_high and lower limit value hist_low.Calculated result, which is sent to, to be turned
Change coefficient calculation part 6.Conversion coefficient calculation part 6 calculates conversion coefficient.
Pixel value converter section 7 is according to the calculated conversion coefficient of conversion coefficient calculation part 6, to the number of reference block and reference block
According to gray scale reduction processing is carried out, the low-resolution image as correlation operand is generated.Pixel value converter section 7 is according to every 1
A pixel is the data of the reference block of 12 bits, generates the low-resolution image that every 1 pixel is 6 bits.Furthermore same, according to
Every 1 pixel is the data of the reference block of 12 bits, generates the low-resolution image that every 1 pixel is 6 bits.
After generating low-resolution image, correlation operational part 3 starts the operation of the correlation of object block.Correlation operation
The result of correlation operation is for example output to device for detecting motion vector etc. by portion 3.
In the above-mentioned after treatment for being directed to a reference block, successively next reference block is similarly handled repeatedly, thus
To 10 overall execution correlation operation of frame data.
Whenever updating reference block, correlation arithmetic unit 100 implements the calculating of histogram.
(effect of embodiment)
The correlation arithmetic unit 100 of present embodiment determines the valid pixel value model comprising characteristic pixel Distribution value
It encloses, and gray scale reduction processing is carried out to the pixel of the valid pixel value range, to generate as the low of correlation operand
Image in different resolution.The low-resolution image for maintaining characteristics of image as much as possible can be used for phase by correlation arithmetic unit 100
Pass, which is worth operation, can implement at a high speed correlation operation so as to cut down the circuit scale of correlation operation.
The correlation arithmetic unit 100 of present embodiment calculates histogram, as pixel Distribution value, will remove close to pixel value
Maximum value (white) and minimum value (black) range other than range be set as valid pixel value range, the maximum close to pixel value
The range of value (white) and minimum value (black) is predicted to be the less range of pixel number.Can according to accurate pixel Distribution value,
Select valid pixel value range.
In addition, the correlation arithmetic unit 100 of present embodiment is a certain number of by removing from valid pixel value range
The biggish pixel of pixel value and the lesser pixel of pixel value, can to noise etc. carry out robust valid pixel value range choosing
It selects.Valid pixel value range can be selected according to accurate pixel Distribution value.
In addition, the data for the reference block that the correlation arithmetic unit 100 of present embodiment is stored according only to block storage,
Calculate the histogram as pixel Distribution value.Therefore, in correlation operation, being able to use is not to maintain the image of reference frame whole
The pixel value distribution trend of body but maintain the low-resolution image of local pixel value distribution trend, so as to basis
Characteristics of image in the image-region of part, finds out the similarity between local image.
In the correlation arithmetic unit 100 of present embodiment, simultaneously column count 144 is configured in correlation operational part 3
In the case where a correlation, compared with the correlation operational part for carrying out correlation operation according to the gray scale of 12 bits, according to 6 bits
Gray scale carry out correlation operation correlation operational part 3 circuit scale can be reduced to it is approximately half of.In order to generate low point
The circuit scale and phase of pixel Distribution value calculation part 5, conversion coefficient calculation part 6 needed for resolution image and pixel value converter section 7
The circuit scale of pass value operational part 3 being cut in is compared to very small.Therefore, 100 energy of correlation arithmetic unit of present embodiment
It is enough to cut down circuit scale on the whole.
(variation)
More than, an embodiment of the invention is illustrated referring to attached drawing, but specific structure is not limited to
The embodiment, the also design alteration etc. comprising not departing from spirit of the scope of the present invention.In addition, said one embodiment and with
Structural element shown in variation shown in lower can be made up of appropriately combined.
(variation 1)
For example, the correlation arithmetic unit 100 of above embodiment is operation difference absolute value and (SAD) as correlation
Circuit, but correlation is without being limited thereto.For example, correlation be also possible to difference quadratic sum (SSD), normalized crosscorrelation (NCC),
Or zero-mean normalized crosscorrelation (ZNCC) etc..
(variation 2)
In the above-described embodiment, block size is 9 × 9, but block size is without being limited thereto.For example, it may be 16x16,15 ×
15, the block sizes such as 5 × 5,3 × 3, and block size as being also possible to 15 × 9.
(variation 3)
In addition, the conversion coefficient calculation part 6 of the correlation arithmetic unit 100 of above embodiment has been used and has mutually been reduced
Offset and shift amount shift, as the conversion coefficient for converting pixel data into defined gray scale, but what is calculated turns
It is without being limited thereto to change coefficient.For example, the base_out [i] [j] and ref_out [i] [j] of the output as pixel value converter section 7 can
To be found out by linear function shown in formula 11 below.
In this case, conversion coefficient calculation part 6 calculates the slope a and intercept b of above-mentioned linear function as conversion coefficient i.e.
It can.In the case where reducing that treated gray scale being set as 6 bit gray scale, as shown in formula 12 below, by intercept b be set as by
The calculated lower limit value hist_low of pixel Distribution value calculation part 5.
B=-hist_low ... formula 12
Then, according to formula 5 same as the above embodiment, valid pixel value range range is calculated.Finally, as following
Formula 13 shown in, treated gray scale (6 bit) and valid pixel value range range is reduced according to gray scale, finds out slope a.
(variation 4)
In addition, the pixel Distribution value calculation part 5 of above embodiment is according to the reference frame as correlation operand, meter
A kind of histogram is calculated, as pixel Distribution value, but the calculation of histogram is without being limited thereto.For example, image data such as
In the case that RGB or YUV is made of multiple elements like that, histogram can be calculated according to each constituent element of image data
Figure.
Fig. 6 is shown in the case where image data is made of each element of RGB in pixel Distribution value calculation part according to RGB
Each and the histogram that generates.Pixel Distribution value calculation part grasps pixel value distribution trend according to each of RGB, and presses
According to each calculating upper limit value and lower limit value of RGB.Conversion coefficient calculation part calculates conversion coefficient according to each of RGB, as
Plain value converter section carries out gray scale reduction processing according to each of RGB to generate low-resolution image.By according to each of RGB
A progress calculating of histogram, the calculating of conversion coefficient, gray scale reduction processing, correlation arithmetic unit 100 can be more suitably
Generation maintains the low-resolution image of characteristics of image, and is used for correlation operation.
(variation 5)
In addition, the pixel Distribution value calculation part 5 of above embodiment will be except the maximum value (white) and minimum close to pixel value
Range other than the range of value (black) is set as valid pixel value range, the maximum value (white) and minimum value (black) close to pixel value
Range be predicted to be the less range of pixel number.But the setting means of valid pixel value range is without being limited thereto.For example,
Position and the number of the peak value of the pixel number in pixel Distribution value can be grasped to set effective pixel coverage.
Fig. 7 shows the histogram that the peak value number of the pixel number in pixel Distribution value is two.As shown in fig. 7, comprising
In the image on the boundary of object etc., the peak value of the pixel number in the pixel Distribution value of histogram is that multiple situations is more.
In the case where peak intervals are opened, the less pixel value of pixel number between peak value enters valid pixel value range, should
Pixel value is comprised in gray scale and reduces in treated gray scale.
The range for the pixel value for selecting pixel number more is as valid pixel value range, and to the valid pixel value range
Pixel carries out gray scale reduction processing, maintains the low-resolution image of characteristics of image as much as possible to generate, is based on the viewpoint,
The setting of valid pixel value range as described above can not be said to be preferred.
Therefore, divide 4 sections again in the range of the upper limit value and lower limit value once set, calculate with highest
Two mutual distances in section of peak value.Exist in the non-conterminous situation in section, and in each section clipped by the section
In the case where pixel number within threshold value, it is judged as there are two peak values, according to each peak-settings upper limit value and lower limit value.
In addition, the number of dividing again of above content is an example, the quantity for setting peak value as it is N number of in the case where, then divide
Segmentation number also can be set to N × 2+1 or more.
Carry out such pixel Distribution value calculation part handled can by the less pixel value range of pixel number reliably from
It is removed in valid pixel range.
Industrial availability
The present invention can be applied to the device comprising following circuit, which finds out the correlation between the specific region in image
Value.
Label declaration
100: correlation arithmetic unit
1: main memory
2: block storage
21: reference block data storage memory
22: reference block data storage memory
3: correlation operational part
10: frame data
11: benchmark frame data
12: referring to frame data
5: pixel Distribution value calculation part
6: conversion coefficient calculation part
7: pixel value converter section
Claims (7)
1. a kind of correlation arithmetic unit, reference block in operation image, in described image or in other images at least
Correlation between one reference block, in the correlation arithmetic unit, comprising:
Pixel Distribution value calculation part calculates the pixel Distribution value of the pixel in described image, and the picture for selecting pixel number more
Element value range and as valid pixel value range；
Conversion coefficient calculation part calculates the conversion system that the pixel of the valid pixel value range is converted to defined gray scale
Number；And
Pixel value converter section, according to the conversion coefficient, to the pixel in the pixel and the reference block in the reference block
Carry out gray scale reduction processing.
2. correlation arithmetic unit according to claim 1, wherein
The pixel Distribution value calculation part is successively removed from the maximum pixel of pixel value according to the calculated pixel Distribution value
The pixel of number as defined in going, or the successively pixel of the defined number of removing from the smallest pixel of pixel value, thus select
The valid pixel value range.
3. correlation arithmetic unit according to claim 2, wherein
The generation rate of the pixel Distribution value calculation part based on noise determines the pixel removed from the valid pixel value range
Number, the generation rate of the noise is estimated according to the shooting condition of described image.
4. correlation arithmetic unit according to any one of claims 1 to 3, wherein
In the case where a pixel of described image is made of multiple elements, the pixel Distribution value calculation part is wanted according to each
Element calculates pixel Distribution value, and selects valid pixel value range according to each element.
5. correlation arithmetic unit described in any one according to claim 1~4, wherein
The pixel Distribution value calculation part according to the position of the peak value of the pixel number in the calculated pixel Distribution value and
Number, the range for the pixel value for selecting pixel number more and as the valid pixel value range.
6. correlation arithmetic unit according to any one of claims 1 to 5, wherein
The pixel Distribution value calculation part calculates the pixel value point only for the pixel in the reference block in described image
Cloth.
7. correlation arithmetic unit according to any one of claims 1 to 5, wherein
The pixel Distribution value calculation part calculates pixel this both sides' pixel in pixel and the reference block in the reference block
The pixel Distribution value.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/JP2017/014631 WO2018189772A1 (en) | 2017-04-10 | 2017-04-10 | Correlation value calculation device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110537202A true CN110537202A (en) | 2019-12-03 |
| CN110537202B CN110537202B (en) | 2022-10-04 |
Family
ID=63792402
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201780089319.8A Active CN110537202B (en) | 2017-04-10 | 2017-04-10 | Correlation value calculation device |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US10977803B2 (en) |
| CN (1) | CN110537202B (en) |
| WO (1) | WO2018189772A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113566981A (en) * | 2021-09-26 | 2021-10-29 | 南京智谱科技有限公司 | Self-adaptive adjusting method and device of infrared detector |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110537202B (en) * | 2017-04-10 | 2022-10-04 | 奥林巴斯株式会社 | Correlation value calculation device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0219070A (en) * | 1988-07-07 | 1990-01-23 | Toshiba Corp | Picture data gradation converter |
| JPH10222669A (en) * | 1997-02-05 | 1998-08-21 | Sharp Corp | Aligning device |
| JP2008154251A (en) * | 2005-03-25 | 2008-07-03 | Mitsubishi Electric Corp | Image processing device, image display device, and image display method |
| JP2008287605A (en) * | 2007-05-18 | 2008-11-27 | Yokogawa Electric Corp | Histogram creation device and gradation setting device |
| US20090091643A1 (en) * | 2005-06-15 | 2009-04-09 | Olympus Corporation | Image-Acquisition Apparatus |
| JP2014027373A (en) * | 2012-07-24 | 2014-02-06 | Olympus Corp | Image processing apparatus, image processing method and image processing program |
| CN104519274A (en) * | 2013-09-27 | 2015-04-15 | 奥林巴斯株式会社 | Image processing apparatus, image processing method |
| US20150235103A1 (en) * | 2014-02-17 | 2015-08-20 | Canon Kabushiki Kaisha | Distance measurement apparatus, imaging apparatus, distance measurement method and program |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE19549500C2 (en) * | 1994-01-21 | 2000-05-25 | Mitsubishi Electric Corp | Movement vector determiner for various prediction methods |
| US6944331B2 (en) * | 2001-10-26 | 2005-09-13 | National Instruments Corporation | Locating regions in a target image using color matching, luminance pattern matching and hue plane pattern matching |
| JP4304528B2 (en) * | 2005-12-01 | 2009-07-29 | ソニー株式会社 | Image processing apparatus and image processing method |
| US20120075436A1 (en) * | 2010-09-24 | 2012-03-29 | Qualcomm Incorporated | Coding stereo video data |
| US9584814B2 (en) * | 2014-05-15 | 2017-02-28 | Intel Corporation | Content adaptive background foreground segmentation for video coding |
| DE112015006619T5 (en) * | 2015-07-24 | 2018-05-09 | Olympus Corporation | Image processing device, image processing method and program |
| CN110447052B (en) * | 2017-04-05 | 2023-03-14 | 奥林巴斯株式会社 | Correlation value calculation device |
| CN110537202B (en) * | 2017-04-10 | 2022-10-04 | 奥林巴斯株式会社 | Correlation value calculation device |
-
2017
- 2017-04-10 CN CN201780089319.8A patent/CN110537202B/en active Active
- 2017-04-10 WO PCT/JP2017/014631 patent/WO2018189772A1/en active Application Filing
-
2019
- 2019-10-02 US US16/590,537 patent/US10977803B2/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0219070A (en) * | 1988-07-07 | 1990-01-23 | Toshiba Corp | Picture data gradation converter |
| JPH10222669A (en) * | 1997-02-05 | 1998-08-21 | Sharp Corp | Aligning device |
| JP2008154251A (en) * | 2005-03-25 | 2008-07-03 | Mitsubishi Electric Corp | Image processing device, image display device, and image display method |
| US20090091643A1 (en) * | 2005-06-15 | 2009-04-09 | Olympus Corporation | Image-Acquisition Apparatus |
| JP2008287605A (en) * | 2007-05-18 | 2008-11-27 | Yokogawa Electric Corp | Histogram creation device and gradation setting device |
| JP2014027373A (en) * | 2012-07-24 | 2014-02-06 | Olympus Corp | Image processing apparatus, image processing method and image processing program |
| CN104519274A (en) * | 2013-09-27 | 2015-04-15 | 奥林巴斯株式会社 | Image processing apparatus, image processing method |
| US20150235103A1 (en) * | 2014-02-17 | 2015-08-20 | Canon Kabushiki Kaisha | Distance measurement apparatus, imaging apparatus, distance measurement method and program |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113566981A (en) * | 2021-09-26 | 2021-10-29 | 南京智谱科技有限公司 | Self-adaptive adjusting method and device of infrared detector |
Also Published As
| Publication number | Publication date |
|---|---|
| US20200034975A1 (en) | 2020-01-30 |
| WO2018189772A1 (en) | 2018-10-18 |
| US10977803B2 (en) | 2021-04-13 |
| CN110537202B (en) | 2022-10-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101281650B (en) | Quick global motion estimating method for steadying video | |
| CN106663326B (en) | Image processing system and computer readable recording medium | |
| US10964000B2 (en) | Techniques for reducing noise in video | |
| CN106851046A (en) | Video dynamic super-resolution processing method and system | |
| US9230307B2 (en) | Image processing apparatus and method for generating a high resolution image | |
| US10818018B2 (en) | Image processing apparatus, image processing method, and non-transitory computer-readable storage medium | |
| CN109118510A (en) | A kind of monitor video processing method, device and computer-readable medium | |
| JP6194450B2 (en) | State estimation device, program, and integrated circuit | |
| KR20100118978A (en) | Sparse geometry for super resolution video processing | |
| JP7032871B2 (en) | Image processing equipment and image processing methods, programs, storage media | |
| Vajgl et al. | Advanced F‐Transform‐Based Image Fusion | |
| CN110537202A (en) | Correlation arithmetic unit | |
| KR20050012768A (en) | Unit for and method of estimating a current motion vector | |
| CN108270945A (en) | A kind of motion compensation denoising method and device | |
| EP1361541B1 (en) | Process and device for global motion estimation in a sequence of images, for instance for optical mice | |
| Li et al. | A full-process optimization-based background subtraction for moving object detection on general-purpose embedded devices | |
| CN101616324A (en) | The blocking effect removal device of compression coded image | |
| US20130301928A1 (en) | Shift vector reliability determining apparatus and method | |
| US6876778B2 (en) | Edge enhancement method and apparatus in digital image scalar-up circuit | |
| Zhang et al. | A ViBe based moving targets edge detection algorithm and its parallel implementation | |
| WO2013031418A1 (en) | Device for detecting line segment and arc | |
| CN116309168A (en) | Low-illumination image enhancement method and system for parallel hybrid attention progressive fusion | |
| CN110447052A (en) | Correlation arithmetic unit | |
| Chang et al. | Real-time Hybrid Stereo Vision System for HD Resolution Disparity Map. | |
| JP4239411B2 (en) | Motion judgment device and motion judgment method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |