JP2002271818A - Parallax amount measurement device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a parallax amount measurement device that can detect correct matching while clearly distinguishing it from error matching at parallax measurement. SOLUTION: The parallax amount measurement device is provided with a measurement block setting means 3 that sets a measurement block on a 1st image 1 of a stereoscopic image, matching means 4, 10 that retrieve a block area corresponding to the measurement block on a 2nd image 2 of the stereoscopic image to detect the best similarity and measure the parallax depending on a difference of coordinates of both the images at the best similarity, a correction data generating means 13 that generates correction data 17 for correcting the best similarity on the basis of a difference absolute value sum of pixel data of corresponding pixels between a measurement block and a peripheral block resulting from shifting the measurement block by prescribed pixels, a correction means 14 that uses the correction data 17 to correct the best similarity, and an error discrimination means 15 that discriminates whether or not matching by the matching means denotes error matching on the basis of the corrected best similarity.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a parallax amount measuring apparatus for measuring a parallax amount of a stereo image.

[0002]

2. Description of the Related Art Conventionally, techniques for measuring the amount of parallax of a stereo image have various fields of application. For example, JP-A-7-
As described in Japanese Patent Publication No. 120255, stereo distance measurement in which a plurality of cameras capture images of a front view and obtain depth information from the amount of parallax has been known for a long time. In this stereo distance measurement, the depth of a specific place may be measured, or the entire parallax may be finely divided, and the depth may be measured for each of them.

[0003] In the field of displaying three-dimensional images for entertainment or business use, there is also a method of adding the measured disparity information to one of the images and transmitting the same to reduce the amount of transmission.
Further, as described in, for example, JP-A-9-322199 and JP-A-4-360394, the amount of parallax of an image is monitored, and the amount of image is adjusted so that the amount of parallax becomes good for the observer. There are also methods of correcting camera settings and camera settings, and switching display conditions as described in JP-A-11-355808. In such a usage, a detailed parallax distribution in the screen may be measured, but a parallax amount of a specific portion, or a statistic such as a mode or a maximum value of the parallax may be used.

[0004] As a method for measuring the amount of parallax, a block matching method has conventionally been often used.
In this method, for example, when measuring the parallax of two left and right (LR) images, the screen of the L image is divided into several measurement blocks of M × N pixels as shown in FIG.
Alternatively, several measurement blocks are designated on the screen of the L image, a corresponding place, that is, the most similar place is searched for on the screen of the R image for each measurement block, and based on the amount of displacement between the two. The parallax amount is measured.

Here, when searching for a corresponding place, as shown in FIG. 14, for example, the coordinates of the measurement block on the L image are set to (XL, YL), and the L
A block B of M × N pixels is set at the same coordinates as the image, and the sum of the luminance differences (absolute values) between the pixels between the blocks A and B according to the following formula (hereinafter referred to as the sum of absolute differences) Is calculated.

[0006]

(Equation 1)

The sum S of absolute differences between the blocks A and B described above
Is minimized at a location where block B corresponds to block A, so that block B is shifted, that is, ΔX, ΔY
Is calculated while changing the value within a proper range to find a location where the sum S is minimum. In this way, when the location of the block B where the sum of absolute differences S is minimized is found, the deviation amounts ΔX and ΔY between the location of the block B and the location of the block A are calculated as the parallax amounts corresponding to the block A. . Here, if it is exactly the same as block A block B, the sum of absolute difference becomes zero, it is rare to be exactly the same for normal stereo image, with a certain value.

[0008]

However, in the ordinary block matching method, for example, when there is not a clear change in luminance in a block, similar portions exist everywhere and slight noise or image There is a case where matching cannot be performed correctly due to the difference. Also, if the image contains a lot of noise, or if the block contains objects with greatly different occlusions and parallaxes, the block has a corresponding place only in a part of the block, Phenomena such as the existence of such a pattern occur, and a matching error is likely to occur. When such a matching error occurs, an abnormal value is included in the measurement result, and a large error occurs in the above-described distance measurement, and correction or change of the display condition becomes inappropriate.

In order to avoid the occurrence of the above-mentioned matching error, there is also known a method of eliminating abnormal information or performing some kind of interpolation processing so as not to have an adverse effect. For example, as a countermeasure for a case where there is not a clear luminance change sufficiently in a block, for example, Japanese Patent Laid-Open Publication No. Hei 7-120255 discloses that there is a high possibility of error matching by checking the presence or absence of an edge, that is, the presence of a clear luminance change. It is disclosed that a determination is made as to whether or not a location having a high possibility is excluded in advance.

[0010] As a countermeasure against a case where an image has a lot of noise or an object having a large difference in occlusion or parallax is included in a block, for example, Japanese Patent Laid-Open No.
JP-A-120255 and JP-A-4-329481 disclose methods of determining error matching and weighting based on the minimum value of the sum of absolute differences.

However, since the sum of absolute differences depends not only on the similarity between blocks but also on the brightness and contrast of the image, it is not always possible to clearly distinguish between an error and a correct answer. it is difficult to. For example, matching and is a set of stereo images, the average brightness depends gain to the brightness in the same pattern as the stereo image is also or Also changed contrast, a stereo image as the contrast the average luminance the same has changed The result is that the sum of absolute differences changes between the two stereo images despite the fact that it is almost unchanged (the same matching error occurs at the same place). the determination, as in the former stereo images has been determined that the OK is or is determined as an error in the latter stereo images, in the latter stereo image which has been determined as an error in the former stereo image reversed OK and there is a case to be or is.

As a method for solving such a problem,
There is known a method of normalizing the minimum value of the sum of absolute differences with the sum or average value of luminance in a measurement block. However, this method is effective for a stereo image in which a gain is applied to luminance, but is not effective for a stereo image in which only the contrast is changed. In addition, in the case of a stereo image in which the brightness of the entire image is offset, the average brightness changes even though the sum of absolute differences does not change, so extra correction is performed. .

Accordingly, an object of the present invention made in view of the above point is that error matching and correct matching in disparity measurement can be clearly separated and detected, and error matching in calculating disparity statistics and the like can be performed. An object of the present invention is to provide a parallax amount measuring device capable of reducing the influence.

[0014]

According to a first aspect of the present invention, there is provided a parallax amount measuring apparatus, comprising: a stereoscopic image comprising a first image and a second image; A measurement block setting means for setting a measurement block; a search for a block area corresponding to the measurement block on the second image to detect a best value of similarity; and a difference between coordinates of the two images at the best value. From the matching means for measuring the amount of parallax of the measurement block, based on the sum of absolute differences of pixel data between corresponding pixels between the measurement block and a peripheral block shifted by a predetermined number of pixels of the measurement block, correcting the correction data generation means for generating correction data for correcting the best degree of similarity, by the correction data generated by the correction data generating means the best value of the similarity A positive means, is characterized in that it has a, and determining the error determining means for determining whether matching error in the matching means based on the best values of the corrected similarity by said correcting means.

According to the parallax amount measuring apparatus according to the first aspect,
The similarity between the measurement block and the block area on the second image is calculated to detect the best value of the similarity, and the difference between the measurement block and a peripheral block obtained by shifting the measurement block by a predetermined pixel is calculated. The correction data is generated by calculating the sum of absolute differences of the pixel data of the corresponding pixels, and the best value of the similarity is corrected based on the correction data, and whether the matching is error based on the best value of the similarity after the correction is performed. Since the determination is made, the reliability of matching can be determined without being affected by changes in luminance or contrast, and the accuracy of parallax measurement can be improved.

[0016] invention parallax amount measuring device according to claim 2,
A measurement block setting means for setting a predetermined measurement block on the first image of the stereo image composed of the first image and the second image; and a block area corresponding to the measurement block on the second image. A matching means for searching for and detecting the best value of the similarity and measuring the amount of parallax of the measurement block from the difference in the coordinates of the two images at the best value; and performing a Laplacian operation on the pixel data of the measurement block, and performing the operation. Correction data generating means for integrating results and generating correction data for correcting the best value of the similarity; and correcting means for correcting the best value of the similarity based on the correction data generated by the correction data generating means. And error determining means for determining whether or not the matching by the matching means is an error based on the best value of the similarity corrected by the correcting means. It is characterized in that.

According to the parallax amount measuring apparatus according to claim 2,
The similarity between the measurement block and the block area on the second image is calculated to detect the best value of the similarity, and the pixel data of the measurement block is subjected to the Laplacian calculation, and the calculation result is integrated to correct the correction data. Is generated, the best value of the similarity is corrected by the correction data, and it is determined whether the matching is an error based on the corrected best value of the similarity. can determine the matching reliability without being affected by changes in brightness or contrast, it is possible to improve the accuracy of parallax measurements.

The invention according to claim 3 is the invention according to claim 1 or 2
In the parallax amount measuring device described in Item 1, the correction means normalizes the best value of the similarity based on the correction data.

According to the parallax amount measuring apparatus of the third aspect,
Since the best value of the similarity is normalized by the correction data in the correction means, it is possible to more clearly separate and detect error matching and correct matching.

[0020]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the parallax amount measuring device according to the present invention will be described with reference to Figs.

(First Embodiment) FIGS. 1 to 7 show a first embodiment. FIG. 1 is a block diagram showing a circuit configuration of a main part of the whole, and FIG. 2 is a parallax measurement shown in FIG. FIG. 3 is a block diagram showing a circuit configuration of a main part of the unit, FIG. 3 is a diagram for explaining a method of generating correction data, FIG. 4 is a block diagram showing a configuration of a main part of the correction data generation circuit shown in FIG. Is a flowchart showing an outline of the parallax amount measuring operation, FIG. 6 is a flowchart showing a detailed operation of the correction data generation processing shown in FIG. 5,
FIG. 7 is a flowchart showing a detailed operation of the matching calculation processing shown in FIG.

As shown in FIG. 1, the parallax amount measuring apparatus according to the present embodiment sends one of the left and right (LR) images 1 and 2 constituting a stereo image, here the L image 1, to the measurement block setting section 3. , And the other R image 2 is supplied to the search range / reference block setting unit 4. In the measurement block setting unit 3,
A predetermined number of measurement blocks are set on the L image 1 based on preset coordinates, and the image data of each measurement block is read into a memory or the like, and pixel data of each measurement block is sequentially output as reference data 5. Further, the search range / reference block setting unit 4 sets a search range centered on the same coordinates (X, Y) as the measurement block on the R image 2 and a reference block having the same size as the measurement block within the search range. Is set, and the pixel data of each reference block is sequentially output as reference data 6 by, for example, reading image data in each reference block into a memory.

The reference data 5 and the reference data 6 are supplied to a parallax measuring section 9 where a matching operation is performed to measure and output the parallax amount 7 of each measurement block, and the parallax amount 7 is measured. It is determined whether or not the matching at this time is an error matching, and an error code 8 representing the result of the determination is output.

As shown in FIG. 2, the parallax measuring section 9 includes a matching calculation circuit 10, a correction data generation circuit 13, a calculation result correction circuit 14, and an error determination circuit 15.

The matching calculation circuit 10 performs a matching calculation process for the measurement block, and has a difference absolute value sum calculation circuit 11 and a minimum value detection circuit 12. sum of absolute differences between the data 5 and the reference data 6, wherein the calculation result of the difference absolute value sum in supplying the minimum value detecting circuit 12 calculates the sum of absolute difference of the luminance, the minimum value detecting circuit 12 until it Compared with the minimum value of, if the value is large, the value is discarded; if the value is small, the minimum value is replaced, and the current shift amount (coordinates X + ΔX, Y + ΔY from which the reference data is cut out,
Center coordinate X of the search range, it substitutes the difference) between the Y.

The shift amount of the series of the reference block operations between the difference absolute value sum computation and the minimum value detection, i.e. by repeating changing the position of cutting out reference data, searches an area corresponding to the measurement block parallax The quantity 7 (ΔX, ΔY) is determined and output, and the minimum value 16 of the sum of absolute differences at that time, that is, the best value of the similarity between the measurement block and the reference block is supplied to the calculation result correction circuit 14.

On the other hand, the correction data generating circuit 13, a reference data 5 based on generating a correction data 17 for correcting the minimum value 16 of the sum of absolute differences is supplied to the operation result correction circuit 14, wherein Divides the minimum value 16 of the sum of absolute differences by the correction data 17, ie, normalizes the minimum value 16 of the sum of absolute differences by the correction data 17, and corrects the corrected data 18.
The resulting, and supplies it to the error determining circuit 15. The error judging circuit 15, an error code 8 data 18 after correction means NG is greater than a predetermined threshold,
If smaller, an error code 8 indicating OK is output.
When calculating the parallax map and the parallax statistics of the entire image, this error code 8 is used if the error code 8 is NG and the corresponding parallax amount is not adopted, or the parallax amount is corrected by another means. It is used for.

Next, the correction data 1 in the present embodiment
Referring to FIG. 3 described 7 generation method. In FIG. 3, the measurement block on the L image is 4 × 4 pixels in order to simplify the drawing. In the present embodiment, an A block, a B block, a C block, and a D block are set by shifting the X block by one pixel in each of the upper, lower, lower, and upper directions. The absolute value of the difference between each pixel and the corresponding pixel in the X block is calculated and summed. That is, the sum of the absolute differences between the reference data of the X block and the data of the A to D blocks shifted by one pixel is calculated, and the sum of the four sums of the absolute differences is used as the correction data 17. Therefore, in this case, the correction data 17 (Cr)
The sum of absolute differences between the D block and the X block is Diff
A, DiffB, DiffC, DiffD can be expressed by the following equation (2).

[0029]

(Equation 2)

Here, in the case of an image in which the luminance of the L and R images is uniformly increased by ΔI, the minimum value S ′ (ΔX, ΔY) of the sum of absolute differences obtained as a result of the matching is expressed by the following (3)
Represented by the formula, the minimum value S of the original image that does not increase the brightness
(ΔX, ΔY). Similarly, the difference absolute value sums Diff'A to Diff'D of the blocks A to D do not change, so that the correction data Cr 'is also the correction data Cr of the original image.
It does not change.

[0031]

(Equation 3)

[0032] By contrast, L, if the difference of the image to n times the average brightness of the brightness and image of each pixel of the R image (Im), i.e. in the case of an image with different contrast the average brightness constant The minimum value S ′ (ΔX, ΔY) of the sum of absolute differences obtained as a result of the matching is obtained by multiplying the minimum value S (ΔX, ΔY) of the original image by n as represented by the following equation (4). It will be.

[0033]

(Equation 4)

The difference absolute value sum Dif of the A block is
f'A also

(Equation 5) Since Diff'B, Diff'C, and Diff'D are each multiplied by n, the correction data Cr 'is also obtained by multiplying the correction data Cr of the original image by n.

Therefore, when S 'is divided by Cr', the minimum value of the corrected sum of absolute differences, which is not affected by the contrast change, can be obtained. it is possible to determine the error. Also, when the luminance of each pixel of the L and R images is increased by n times, since S 'and Cr' are simply increased by n times, S '/
Cr 'is not affected by a change in luminance, and an error can be clearly determined by comparing it with a specified value.

[0036] FIG. 4 is a block diagram showing a configuration of a main part of the correction data generating circuit 13 which generates a correction data as described above. The correction data generation circuit 13 has a delay unit 19 and a calculation unit 26. The delay unit 19 has one-line delay circuits 20 and 21 including memories and the like, and flip-flops (FFs) 22, 23, 24 and 25.
The block data (D data) 36 is obtained. Also, the reference data 5 in the one-line delay circuit 20 delays by one line to obtain data (C data) 35 of the C blocks, X block data delayed the one line delays one clock by the flip-flop 24 of data (X data) 34
And the data is delayed by two clocks by the flip-flop 23 to obtain the data (B data) 33 of the B block.
Further, the reference data 5 is delayed by a total of two lines by the one-line delay circuits 20 and 21, and the flip-flop 22
To obtain the data (A data) 32 of the A block by delaying by one clock.

[0037] calculation unit 26 includes a difference circuit 271 and absolute value circuit 272 corresponds to the A data, a differential circuit 281 and absolute value circuit 282 which corresponds to the B data, the difference circuit 291 and absolute value circuit corresponding to the C data 292,
Difference circuit 301 and absolute value circuit 3 corresponding to D data
And 02, and a summing circuit 311 and 312, and a holding circuit 313. Difference circuits 271, 281, 291, 30
1 supplies the X data 34 from the delay unit 19 and supplies the corresponding A to D data 32, 33, 35, and 36, and calculates the difference between the corresponding pixels of the X block and the A to D blocks. determined, the absolute value circuit corresponding to their difference 27
2, 282, 292, and 302 convert the data to an absolute value and add it by an adder circuit 311. The adder 312 and the holding circuit 313 accumulate the added output regarding the sequential pixels output from the adder circuit 311. In this way, obtaining the correction data 17 by performing the calculation of equation (2).

FIG. 5 is a flowchart showing an outline of the parallax amount measuring operation according to the above-described embodiment. First,
Set the measurement block to one image of the stereo image (step S1), and after performing the correction data generating process (step S2) and matching processing (step S3), and the sum of absolute differences obtained in the matching operation Is divided by the correction data to correct, that is, normalize, the operation result (step S4).

Thereafter, it is determined whether or not the minimum value (corrected data) of the normalized sum of absolute differences is equal to or smaller than a specified value (step S5). Here, if the corrected data is equal to or less than the specified value and the determination result is YES, it is determined that the matching is OK (step S6). If NO, that is, if the corrected data exceeds the specified value, the matching is performed. It is determined as NG (step S7), and the parallax amount obtained by the matching calculation is output together with the error determination result (step S7).
8).

FIG. 6 is a flowchart showing the detailed operation of the correction data generation processing in step S2 of FIG. First, the A block in which the measurement block is shifted by -1 pixel in the y direction, the B block in which the measurement block is shifted by -1 pixel in the x direction, the C block in which the measurement block is shifted by +1 pixel in the x direction, and the measurement block in the y direction by +1 Each of the D blocks whose pixels have been shifted is set (step S
11 to S14), and calculate the sum of absolute values of the differences between the pixels in the A to D blocks and the corresponding pixels in the measurement block (steps S15 to S18). Thereafter, the calculated sums of the absolute values of the differences are totaled to obtain correction data (step S19).

FIG. 7 is a flowchart showing the detailed operation of the matching calculation processing in step S3 of FIG. First, R corresponding to the measurement block set to the L image
Within the predetermined search range on the image, the reference data setting process (step S21) and the difference absolute value sum calculation process with the measurement block (step S22) are sequentially performed, and the search is performed while comparing the sequential difference absolute value sums. The minimum value of the sum of absolute differences within the range is detected (step S23), and the parallax amount at the minimum value is output (step S24).

According to the present embodiment, A to D blocks are set by shifting the measurement block by one pixel in the upper, lower, right and lower directions, respectively, and the data of the A to D blocks shifted by one pixel from the reference data of the measurement block are set. and respectively calculates the difference absolute value sum of obtains the correction data 17 by summing their four difference absolute value sum, the minimum value of the sum of absolute differences obtained by the matching processing by the correction data 17 (similar Since the matching error is determined based on the normalized corrected data, the average brightness and the contrast are changed due to the gain applied to the brightness, or the average brightness is changed. Even if the contrast is the same or the contrast is changed, the matching error can be clearly determined.

(Second Embodiment) FIGS. 8 and 9 show a second embodiment.
Shows an embodiment, FIG. 8 is a diagram for explaining a method of generating correction data 17, FIG. 9 is the correction data generating circuit 1
3 is a block diagram showing a configuration of a main part of the delay section 19 in.

In this embodiment, as shown in FIG.
A to D blocks are set by shifting the block (measurement block) by one pixel in each of the oblique upper and lower directions of ± 45 degrees, and the difference in luminance between each pixel in the A to D blocks and the corresponding pixel in the X block The sum of absolute values is calculated, and the sum of the four absolute differences is summed to obtain correction data 17.

For this reason, as shown in FIG. 9, the delay unit 19 of the correction data generation circuit 13 includes one-line delay circuits 41 and 42 including a memory and the like, and a flip-flop (FF) 4
3, 44, and 45. The reference data 5 is output as it is to obtain data D of the D block (D data) 36. The reference data 5 is delayed by two clocks by the flip-flop 45 to obtain the data of the C block. Data (C data) 35 is obtained. Also, the reference data 5 is delayed one line by one line delay circuit 41, is further delayed by one clock by the flip-flop 44 of the X-block data (X data) 34
Get. Further, the reference data 5 is transferred to the one-line delay circuit 4
The data (B data) 33 of the B block is obtained by delaying two lines in total at 1 and 42, and the data (A data) 32 of the A block is obtained by further delaying the data by two clocks by the flip-flop 43. Other configurations and operations
This is the same as in the first embodiment.

In this way, the A to D blocks are set by shifting the measurement block by one pixel in the up and down directions of ± 45 degrees, respectively, and the reference data of the measurement block and the A to D blocks which are shifted by one pixel diagonally are set. The sum of absolute differences with the data is calculated, and the sum of the four absolute differences is summed to obtain correction data 17. The minimum value of the sum of absolute differences (the sum of the differences) obtained in the matching calculation process using the correction data 17 is obtained. If the matching error is determined based on the normalized data after the correction (similarity best value) 16, as in the first embodiment, the luminance is multiplied by the gain and the average is obtained. Even if the brightness and contrast are different, or the average brightness is the same but the contrast is changing,
The matching error can be clearly determined.

(Third Embodiment) FIGS. 10 and 11 show a third embodiment. FIG. 10 is a block diagram showing a configuration of a main part of a correction data generating circuit 13, and FIG.
FIG. 4 is a diagram for explaining a 3 × 3 Laplacian filter at 0.

In the present embodiment, the measurement block shown in FIG.
A 3 × 3 Laplacian filter as shown in FIG. 1 is applied, and the result is integrated in a block to obtain correction data. Therefore, as shown in FIG. 10, the correction data generation circuit 13 includes a 3 × 3 Laplacian filter operation unit 50 that applies a 3 × 3 Laplacian filter to the reference data 5 and integrates the operation result to perform correction. configuring and a integrator 52 to obtain the data 17.

3 × 3 Laplacian filter operation unit 50
Is for example a one-line delay circuit 501 and 502, a flip-flop 503, 504, 505, 506 and multiplier 507,508,509,510,511, constructed and an adding circuit 512, the reference data 5 Is delayed by one clock by the flip-flop 506,
In supplied to the adding circuit 512 by a factor -1. Further, the reference data 5 is delayed by one line by a one-line delay circuit 501, the data delayed by one line is multiplied by −1 by a multiplier 510 and supplied to an adder circuit 512, and the data is delayed by one by a flip-flop 505. after clock delay, multiplier 50
9 and supplies it to the adder circuit 512, and further delays it by two clocks in the flip-flop 504.
Supplied to the adding circuit 512 8 -1 multiplied by. Further, the reference data 5 is added to the one-line delay circuits 501 and 502 for a total of 2
After the delay by the line, the signal is delayed by one clock by the flip-flop 503, and further multiplied by −1 by the multiplier 507 and supplied to the addition circuit 512. The addition circuit 512 adds the output of multiplier 507,508,509,510,511, supplies the addition output to the integrator 52.

The integrator 52 includes an absolute value circuit 521, an adder circuit 522, and a holding circuit 523, and has a 3 × 3
The output of the Laplacian filter operation unit 50 is used as an absolute value circuit 5.
The data is converted into an absolute value at 21, integrated by accumulating at the adding circuit 522 and the holding circuit 523, and the total value (integrated value) is output as the correction data 17 from the holding circuit 523. Other configurations and operations are the same as the first embodiment.

In the case of the present embodiment, the correction data 17 is expressed as follows.

[0052]

(Equation 6)

The above equation (6) is different from the equation (2) explained in the first embodiment. However, since the change in contrast or luminance is the same as the sum of the absolute differences of the matching results, the correction is not performed. possible to become. In addition, the present embodiment is more suitable for stricter determination, since the tendency of determining an error in a portion having a lot of gradation or blur becomes stronger than that of the first embodiment.

It should be noted that the present invention is not limited to the above-described embodiment, and various modifications or changes can be made. For example, in the first and second embodiments, when obtaining the correction data 17, the A to D blocks in which the measurement block (X block) is shifted in four directions are used, but the measurement block, the A block, and the D block are used. each difference absolute value sum DiffA and DiffD, each difference absolute value sum DiffB and DiffC the measurement block and the B block and C blocks a, does not change so much value, operation corresponding to the C block and the D block it is also possible to dispense with.

Further, by performing preprocessing such as resolution (number of pixels) reduction and smoothing on the input image, it is possible to reduce the measurement time and the circuit scale and to reduce the influence of noise. Furthermore, as it is normally done, it is also possible to advance by edge detection filter, to check whether there are features only available in Matchishigu in the block, so as to omit the if insufficient measurement. However, in the present invention, not only to determine the error in the magnitude of the difference absolute value sum of the matching result, since the normalized sum of absolute difference in values associated with the edge of the block, skip the dry-characteristic check It is also possible to do. Furthermore, the matching operation is not limited to the case of using the luminance of each pixel between blocks, and matching operation by color using Color intensity may be one that integrates the results of the respective colors.

Next, an application example of the parallax amount measuring apparatus according to the present invention will be described with reference to FIG. FIG. 12 shows a case where the present invention is applied to a head mounted display (HMD). This HMD is, the video display controller 53, HMD
Unit 54, the preprocessing unit 55, evaluation value calculation unit 56, an evaluation unit 57 and a control signal output unit 58, the measurement block setting unit 3 is described in the above embodiment, the search range / reference block setting unit 4 and the parallax measuring And a parallax amount measuring device including a unit 9.

In FIG. 12, a stereo video signal is input to a video display controller 53 and displayed on the HMD unit 54 under the control of the video display controller 53. Here, the stereo video signal is, for example, a video signal of a game machine, a video, a computer, or the like. The left and right video may be transmitted alternately for each field, or the left and right video may be included in one field. May be available.

The HMD unit 54 includes two display units corresponding to the left and right eyes of the user, and displays a left image on the left eye display unit and a right image on the right eye display unit, respectively. Users can watch stereoscopic stereo images. In addition, by controlling the video display controller 53 to display only one of the left and right images on both the left and right display units, the user can view a normal 2D video. If the input video is not a stereo video but a normal 2D video, the user can view the normal 2D video by displaying the input video signal on both the left and right display sections as it is. . Such display switching may be performed by the user manually switching the image display controller 53 by providing a changeover switch, or by automatically detecting the type of the video signal.

The stereo video signal is also input to the pre-processing unit 55, where the left and right images are stored in respective memories at a specific timing, and subjected to pre-processing such as resolution conversion and smoothing for parallax measurement. Then, as described in the above embodiment, the measurement block setting unit 3 sets a plurality of measurement blocks on the left image, and the search range / reference block setting unit 4 sets each measurement block from the right image. A corresponding reference block area is set, and the parallax measuring unit 9 performs block matching to measure the parallax amount 7 and outputs an error code 8 indicating whether the matching is correct or not.

The parallax amount 7 and the error code 8 output from the parallax measuring unit 9 are input to the evaluation amount calculating unit 56, where the results of the parallax measurement for the measurement blocks for one screen are integrated to calculate the evaluation amount. to. For example, after calculating the statistic of the amount of parallax in the screen (maximum value, mode value, average value, etc.), an evaluation value for the image is given according to a predetermined rule, and the evaluation value is used as the evaluation value up to that time. to add. Here, the evaluation value of the image is obtained in accordance with a predetermined rule such as squaring by multiplying the maximum value of the parallax by a coefficient or referring to a table for ranking according to statistics. When calculating the statistic, the error code 8 is referred to, and if the code indicates NG, the amount of parallax at that time is ignored.

[0061] evaluation value accumulated in the evaluation quantity calculating unit 56,
The evaluation unit 57 compares the value with the specified value.
By sending an instruction signal of the display switching to forcibly switch the image display from the stereoscopic display to the 2D image display in the image display controller 53.

The advantages of the HMD performing such control will be described. In general, stereoscopic images are difficult to see if the amount of parallax is too large, or the stereoscopic effect is impaired.Therefore, it is desirable to view stereoscopic images with an appropriate parallax suitable for the display device. The parallax of the displayed image may be inappropriate on another display device. Also,
In recent years, there are some devices that convert a 2D video or a game video into a stereo video. However, if the setting is inappropriate, a video having a large amount of parallax may be continuously displayed. Therefore, the amount of parallax included in the video is monitored, and if a large amount of parallax continues, it is highly likely that the setting of the video is inappropriate. For example, by switching stereo display to 2D display, the user does not notice. It is possible to avoid watching the improperly displayed video as it is.

In the above application example, when the accumulated evaluation value exceeds the specified value, the control signal output unit 58 sends a display switching instruction signal to the video display controller 53. Instead of the switching instruction, a signal for displaying a message prompting the user to confirm the parallax setting can be output. Further, for example, the video display can be controlled so that the mode value of the parallax amount becomes an appropriate value. In this case, the read position of the video signal may be shifted left and right, and the left and right video displays may be shifted in opposite directions. In the evaluation amount calculation, the parallax at the error location is ignored. However, when there are a large number of error locations, a specific value may be applied to the evaluation amount.

[0064]

As described above, according to the present invention, the best value of the similarity is detected by calculating the similarity between the measurement block and the block area on the second image. Calculate the sum of the absolute differences of the pixel data of the corresponding pixels between the peripheral block obtained by shifting the block by a predetermined pixel, or perform the Laplacian operation on the pixel data of the measurement block, and integrate the operation result. The correction data is generated, the best value of the similarity is corrected based on the correction data, and it is determined whether or not the matching is an error based on the corrected best value of the similarity. Error matching and correct matching can be clearly separated and detected, and the influence of error matching can be reduced when calculating the statistics of parallax and the like.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a circuit configuration of a main part of an entire parallax amount measuring apparatus according to a first embodiment of the present invention.

FIG. 2 is a block diagram illustrating a circuit configuration of a main part of a parallax measurement unit illustrated in FIG. 1;

FIG. 3 is a diagram for explaining a method of generating correction data according to the first embodiment.

4 is a block diagram showing a configuration of a main part of the correction data generating circuit shown in FIG.

FIG. 5 is a flowchart illustrating an outline of a parallax amount measurement operation according to the first embodiment.

6 is a flowchart showing a detailed operation of the correction data generation processing shown in FIG.

7 is a flowchart illustrating the detailed operation of the matching processing shown in FIG.

FIG. 8 is a diagram for explaining a method of generating correction data in a parallax amount measuring apparatus according to a second embodiment of the present invention.

FIG. 9 is a block diagram illustrating a configuration of a main part of a delay unit of a correction data generation circuit according to a second embodiment.

FIG. 10 is a diagram showing a configuration of a main part of a correction data generation circuit in a parallax amount measuring apparatus according to a third embodiment of the present invention.

FIG. 11 is a diagram illustrating a 3 × 3 Laplacian filter according to a third embodiment.

[12] The parallax measuring apparatus according to the present invention is a block diagram showing an example of application to a head-mounted display device.

FIG. 13 is a diagram for explaining a parallax amount measuring method by block matching.

FIG. 14 is also a diagram for explaining a parallax amount measuring method by block matching.

[Explanation of symbols]

 3 measurement block setting unit 4 search range / reference block setting unit 9 parallax measuring unit 10 matching calculation circuit 11 difference absolute value sum calculating circuit 12 the minimum value detecting circuit 13 correction data generating circuit 14 operation result correction circuit 15 error decision circuit 19 the delay unit 26 arithmetic unit 50 3 × 3 Laplacian filter arithmetic unit

Front page of the continued F-term (reference) 2F065 AA06 AA53 EE00 FF05 JJ03 JJ05 JJ26 QQ00 QQ03 QQ11 QQ12 QQ13 QQ14 QQ23 QQ24 QQ25 QQ27 QQ29 QQ32 QQ33 QQ38 QQ41 QQ42 SS13 2F112 AC06 CA08 FA03 FA07 FA21 FA27 FA29 FA38 FA41 5B057 AA20 CA12 CA16 CB13 CB16 CC03 CH01 DA07 DB03 DC02 DC32 5C061 AB12 AB18 AB21

Claims (3)

[Claims] 1. A measurement block setting means for setting a predetermined measurement block on the first image of a stereo image composed of a first image and a second image; and a measurement block on the second image. A matching means for searching the corresponding block area to detect the best value of the similarity, and measuring the parallax amount of the measurement block from the difference in the coordinates of the two images at the best value; Correction data generating means for generating correction data for correcting the best value of the similarity based on a sum of absolute differences between pixel data of corresponding pixels with a peripheral block shifted by a predetermined pixel; and Correcting means for correcting the best value of the similarity based on the correction data generated by the generating means; and matching based on the best value of the similarity corrected by the correcting means. Parallax measuring device matching in grayed means and having a an error determining means for determining whether or not an error. 2. A measurement block setting means for setting a predetermined measurement block on the first image of a stereo image composed of a first image and a second image; and a measurement block on the second image. Matching means for searching for a corresponding block area to detect the best value of similarity and measuring the amount of parallax of the measurement block from a difference in coordinates between the two images at the best value; and Laplacian pixel data of the measurement block. Calculate,
By integrating the result of the operation, the correction data generation means for generating correction data for correcting the best value of the similarity, by the correction data generated by the correction data generating means for correcting the best value of the similarity A parallax amount measuring device, comprising: a correcting unit; and an error determining unit that determines whether or not the matching by the matching unit is an error based on the best value of the similarity corrected by the correcting unit. 3. The parallax amount measuring apparatus according to claim 1, wherein the correction unit normalizes the best value of the similarity based on the correction data.