JP2012222744A - Stereo image processing device, stereo image processing method, and stereo image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide technology which, even when a wide angle lens is used, cannot just obtain an excellent image having the feel of sharpness even on the peripheral part thereof, but also is high in distance measurement accuracy.SOLUTION: A stereo image processing device comprises two sets of video signal output units having photographic lenses 101 and 102 and image pickup elements 103 and 104, a signal processing circuit 105, a peripheral resolution correction circuit 106, and a distance measurement circuit 107. Video signals from the video signal output units are fed into the signal processing circuit 105. The signal processing circuit 105 processes these video signals before being output to the peripheral resolution correction circuit 106. The peripheral resolution correction circuit 106, which is provided corresponding one for one to the plural video signal output units, adds correction to the video signals which are the output signals from the video signal output units to increase the clarity thereof. The distance measurement circuit 107 calculates each pixel, the subject, and the distance in a shot image on the basis of plural video signals having corrections added thereto by the vide signal correction units to find distance measured values.

Description

  The present invention relates to a stereo image processing technique using stereo matching.

  Stereo cameras are known as means for obtaining a stereoscopic image with a sense of presence. In conventional stereo cameras, imaging is performed by moving the position of a focus lens in order to focus a plurality of photographing lenses.

  FIG. 7 is a block diagram showing a schematic configuration of a conventional stereo camera. In the figure, reference numerals 1001 and 1002 are photographing lenses, and reference numerals 1010 and 1011 are focus lenses. The taking lenses 1001 and 1002 have the same lens configuration, and both can perform macro photography from infinity.

  Image pickup elements 1003 and 1004 are provided in front of the photographing lenses 1001 and 1002, and signals obtained by photographing are subjected to A / D conversion processing by an image signal input circuit 1005. The signal after the A / D conversion processing is output to the image processing circuit 1006 and the focus evaluation circuit 1007. The image processing circuit 1006 performs image quality correction such as γ correction, and the focus evaluation circuit 1007 calculates a high-frequency component contrast component of the subject. By extracting, the sharpness is determined.

  The system controller 1008 drives the motor driver 1009 so that the sharpness determined by the focus evaluation circuit 1007 increases. For example, the lens position is adjusted so that the focus lens 1010 can obtain a sharp feeling at a long distance, while the focus lens 1011 can obtain a sharp feeling at a short distance. Then, by moving the other focus lens to the position of the focus lens having a higher sharpness, a stereo image having an excellent sharpness is obtained.

  Also, when the arrangement and angle of the two cameras are fixed, the spatial position and distance of the subject is specified as the intersection of the straight line from one camera to the subject and the straight line from the other camera to the subject. can do. A stereo matching method is known as one of three-dimensional shape restoration methods. In this method, two cameras having a relative positional relationship are arranged on the left and right, a set of images taken with these cameras is used, and an image taken with one camera is taken with the other camera. This is a method of determining which part of a photographed image corresponds to and estimating the three-dimensional position of each point based on the corresponding relationship.

JP 2005-70077 A

  In order to improve the accuracy of pattern matching for obtaining the corresponding points of other images with respect to the reference image in stereo distance measurement, both images are searched so that the corresponding points of the other images with respect to the contour position of the reference image can be searched with high accuracy. It is desirable that the density gradient and the image level of the image of the contour portion are similar.

  When an image having a high resolution and a sharp outline is taken according to the conventional technique, the density gradient and image level of the outline portion of the other image are similar to those of the reference image. For this reason, it is possible to perform pattern matching between images with high accuracy.

  However, in the conventional technique, when a wide-angle lens is used as a photographing lens, the following inconvenience occurs. In the wide-angle lens, the resolution of the peripheral portion is lower than the central portion of the lens. As a result, only the captured image portion at the center of the lens is sharpened. Here, assuming a camera configured to obtain an image with a wide field of view (for example, a monitoring image) by arranging the optical axes of two wide-angle lenses non-parallel to each other, the image is taken at the center of one wide-angle lens. The subject may be photographed at the periphery of the lens with the other wide-angle lens. At this time, since the image taken at the periphery of the wide-angle lens is likely to be blurred, the two subject images obtained from each of the two wide-angle lenses may not be similar in density gradient and image level in the contour portion. Arise. When stereo distance measurement is performed in such a state, the positions of the corresponding points of the image contour vary, and the distance measurement results vary and the accuracy of pattern matching decreases.

  FIG. 8 is a diagram for explaining the above-described problems of the related art with a specific example. FIG. 8A is a diagram showing the relationship between the viewing angle and the subject position of the star mark when the optical axes of the focus lens 1010 and the focus lens 1011 that are wide-angle lenses are arranged non-parallel. FIGS. 8B and 8C are a diagram in which the field of view of the focus lens 1010 and a field of view of the focus lens 1011 are extracted from those shown in FIG. 8A, respectively. .

  As shown in FIG. 8A, the angle of view can be widened by using two wide-angle lenses. In this case, the angle of view is 180 degrees. An image with an angle of view of 180 degrees can be obtained by synthesizing the images output from the image sensor 1003 and the image sensor 1004 by α blending or the like for the portion where the fields of view of both focus lenses overlap, and by fitting the images that do not overlap the fields of view as they are. Is possible. When a wide-angle lens with a reduced resolution at the peripheral portion is used, the peripheral portion of the image after synthesis is blurred.

  As shown in FIG. 8B, the optical axis is inclined by θ in the field of view of the focus lens 1010. According to this figure, since the position of the subject is close to the center of the lens, an image with a sharp density gradient can be obtained.

  On the other hand, as shown in FIG. 8C, in the field of view of the focus lens 1011, the optical axis is inclined by θ in the opposite direction to that of the focus lens 1010. According to this figure, since the position of the subject is located in the periphery of the lens, a blurred image with a gentle density gradient is obtained.

  Thus, even when the same subject is photographed, the sharpness of the photographed image differs depending on the disposition of the wide-angle lens. In other words, there are cases where the density gradient and image level of the contour are different, and the position of the contour is unclear in the blurred image, and the position of the corresponding point varies and is not constant, with the result that the distance measurement result varies. As a result, the accuracy of pattern matching decreases.

  The present invention has been made in view of such problems of the prior art, and the object of the present invention is to use the peripheral portion even when a wide-angle lens that reduces the resolution of an image photographed at the peripheral portion is used. In addition to obtaining an excellent image with a sharpness in the image taken by the camera, it is to provide a technique with high distance measurement accuracy.

In order to solve the above-described problem, a stereo image processing apparatus according to a first aspect of the present invention includes a plurality of video signal output units including a lens and an image sensor that forms a pair with the lens,
A plurality of video signal correction units provided in association with each of the plurality of video signal output units, for correcting and sharpening a video signal that is an output signal from the video signal output unit, and the video A plurality of video signal outputs, each of which includes a distance measuring unit that calculates a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of video signals corrected by each of the signal correction units; The lenses of the same part are within the angle of view and the optical axes are arranged non-parallel to each other, and each of the video signal correction units is for correcting the video signal output from the video signal output unit. In response to the calculation result of the correction high-frequency signal generation unit for generating the high-frequency signal and the distance measurement unit, the variation in the distance measurement value from the distribution of the distance measurement values of each pixel included in the specific area in the imaging Ranging value variation to evaluate Includes a value portion, and an amplitude adjusting section for adjusting an amplitude of the correction frequency signal as the variation of the measured distance becomes smaller.

The stereo image processing method according to the first aspect of the present invention is a plurality of video signal output units including a lens and an imaging device paired with the lens, wherein the lenses of the plurality of video signal output units are A plurality of video signal output units in which the same subject falls within an angle of view and optical axes are arranged non-parallel to each other, and the video signal output units are associated with each of the video signal output units. A method of processing a stereo image using a stereo image processing apparatus having a plurality of video signal correction units for correcting and sharpening a video signal that is an output signal from an output unit, each of the following Has steps.
A distance measuring step for calculating a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of image signals corrected by each of the image signal correction units:
A correcting high-frequency signal generating step for generating a correcting high-frequency signal for the video signal output from the video signal output unit:
Ranging value variation evaluation step of receiving the calculation result of the ranging step and evaluating the variation of the ranging value from the distribution of the ranging value of each pixel included in the specific area in the imaging:
An amplitude adjustment step of adjusting the amplitude of the correction high-frequency signal so that the variation of the distance measurement value is reduced:

  With the configuration of the first aspect, by defining a specific area for obtaining the distance distribution, the number of distance measurement evaluation samples can be increased to improve the evaluation accuracy of the distance variation, and the variation in distance measurement can be suppressed. Since the video signal is corrected, the ranging accuracy can be improved. In particular, by performing distance evaluation on images of the same frame and correcting video signals for each frame, the resolution in the time direction of distance measurement can be increased.

  A stereo image processing apparatus according to a second aspect of the present invention is associated with each of a plurality of video signal output units including a lens and an image sensor paired with the lens, and the plurality of video signal output units. A plurality of video signal correction units provided to correct and sharpen a video signal that is an output signal from the video signal output unit, and a plurality of videos corrected by each of the video signal correction units A distance measuring unit that calculates a distance value by calculating a distance between each pixel in the image pickup and the subject based on the signal, and the lenses of the plurality of video signal output units include the same subject within the angle of view. And the optical axes are arranged non-parallel to each other, and each of the video signal correction units generates a correction high-frequency signal generation unit that generates a correction high-frequency signal for the video signal output from the video signal output unit; Calculation of the distance measuring unit In response to the results, a distance value variation evaluating unit that evaluates the dispersion of distance values based on the distribution of distance values of specific pixels in the imaging in a plurality of frames, and the variation of the distance values is reduced. An amplitude adjustment unit that adjusts the amplitude of the correction high-frequency signal.

The stereo image processing method according to the second aspect of the present invention is a plurality of video signal output units including a lens and an imaging device paired with the lens, wherein the lenses of the plurality of video signal output units are A plurality of video signal output units in which the same subject falls within an angle of view and optical axes are arranged non-parallel to each other, and the video signal output units are associated with each of the video signal output units. A method of processing a stereo image using a stereo image processing apparatus having a plurality of video signal correction units for correcting and sharpening a video signal that is an output signal from an output unit, each of the following Has steps.
A distance measuring step for calculating a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of image signals corrected by each of the image signal correction units:
A correcting high-frequency signal generating step for generating a correcting high-frequency signal for the video signal output from the video signal output unit:
Ranging value variation evaluating step for receiving the calculation result of the ranging step and evaluating the variation of the ranging value from the distribution of ranging values of specific pixels in the imaging in a plurality of frames:
An amplitude adjustment step of adjusting the amplitude of the correction high-frequency signal so that the variation of the distance measurement value is reduced:

  With the configuration of the second aspect, the number of samples increases as the number of shooting frames increases, and the accuracy of evaluating the variation in distance can be improved, and the video signal is corrected so that the variation in distance measurement is suppressed. Therefore, the ranging accuracy can be improved. In particular, it is possible to improve the ranging accuracy with a high spatial resolution.

  A stereo image processing apparatus according to a third aspect of the present invention is associated with each of a plurality of video signal output units including a lens and an image sensor paired with the lens, and the plurality of video signal output units. A plurality of video signal correction units provided to correct and sharpen a video signal that is an output signal from the video signal output unit, and a plurality of videos corrected by each of the video signal correction units A distance measuring unit that calculates a distance value by calculating a distance between each pixel in the image pickup and the subject based on the signal, and the lenses of the plurality of video signal output units include the same subject within the angle of view. And the optical axes are arranged non-parallel to each other, and each of the video signal correction units generates a correction high-frequency signal generation unit that generates a correction high-frequency signal for the video signal output from the video signal output unit; Calculation of the distance measuring unit And a distance measurement value variation evaluation unit that evaluates the dispersion of the distance measurement value from the distribution of distance measurement values of each pixel included in the specific area in the imaging, and the optical axis of the lens of each pixel. A distance calculation unit that calculates a distance from the distance, and an amplitude adjustment unit that adjusts the amplitude of the high-frequency signal for correction based on the distance calculated by the distance calculation unit and variations in the distance measurement value.

The stereo image processing method according to the third aspect of the present invention is a plurality of video signal output units including a lens and an image sensor paired with the lens, wherein the lenses of the plurality of video signal output units are A plurality of video signal output units in which the same subject falls within an angle of view and optical axes are arranged non-parallel to each other, and the video signal output units are associated with each of the video signal output units. A method of processing a stereo image using a stereo image processing apparatus having a plurality of video signal correction units for correcting and sharpening a video signal that is an output signal from an output unit, each of the following Has steps.
A distance measuring step for calculating a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of image signals corrected by each of the image signal correction units:
A correcting high-frequency signal generating step for generating a correcting high-frequency signal for the video signal output from the video signal output unit:
Ranging value variation evaluation step of receiving the calculation result of the ranging step and evaluating the variation of the ranging value from the distribution of the ranging value of each pixel included in the specific area in the imaging:
A distance calculating step for calculating the distance of each pixel from the optical axis of the lens:
An amplitude adjustment step of adjusting the amplitude of the correction high-frequency signal based on the distance calculated in the distance calculation step and the variation of the distance measurement value:

  A stereo image processing apparatus according to a fourth aspect of the present invention is associated with each of a plurality of video signal output units including a lens and an image sensor paired with the lens, and the plurality of video signal output units. A plurality of video signal correction units provided to correct and sharpen a video signal that is an output signal from the video signal output unit, and a plurality of videos corrected by each of the video signal correction units A distance measuring unit that calculates a distance value by calculating a distance between each pixel in the image pickup and the subject based on the signal, and the lenses of the plurality of video signal output units include the same subject within the angle of view. And the optical axes are arranged non-parallel to each other, and each of the video signal correction units generates a correction high-frequency signal generation unit that generates a correction high-frequency signal for the video signal output from the video signal output unit; Calculation of the distance measuring unit In response to the results, a distance value variation evaluation unit that evaluates the dispersion of distance values from the distribution of distance values of specific pixels in the imaging in a plurality of frames, and the distance of each pixel from the optical axis of the lens And a distance adjusting unit that adjusts the amplitude of the high-frequency signal for correction based on the distance calculated by the distance calculating unit and the variation of the distance measurement value.

The stereo image processing method according to the fourth aspect of the present invention is a plurality of video signal output units including a lens and an image sensor paired with the lens, wherein the lenses of the plurality of video signal output units are A plurality of video signal output units in which the same subject falls within an angle of view and optical axes are arranged non-parallel to each other, and the video signal output units are associated with each of the video signal output units. A method of processing a stereo image using a stereo image processing apparatus having a plurality of video signal correction units for correcting and sharpening a video signal that is an output signal from an output unit, each of the following Has steps.
A distance measuring step for calculating a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of image signals corrected by each of the image signal correction units:
A correcting high-frequency signal generating step for generating a correcting high-frequency signal for the video signal output from the video signal output unit:
Ranging value variation evaluating step for receiving the calculation result of the ranging step and evaluating the variation of the ranging value from the distribution of ranging values of specific pixels in the imaging in a plurality of frames:
A distance calculating step for calculating the distance of each pixel from the optical axis of the lens:
An amplitude adjustment step of adjusting the amplitude of the correction high-frequency signal based on the distance calculated in the distance calculation step and the variation of the distance measurement value:

  With the configuration of the third and fourth aspects, the sharpness around the lens is improved, so that the ranging accuracy is improved and an image with good image quality can be obtained simultaneously.

  According to the present invention, it is possible to perform control for sharpening an image from the lens optical axis to the peripheral portion even when a wide-angle lens whose peripheral portion resolution is lowered is used. As a result, the density gradient and image level of the contour of the subject photographed with the photographing subject at the center of the lens can be made similar to the density gradient and image level of the contour of the subject photographed at the lens peripheral portion of one lens. . As a result, the variation in the position of corresponding points in pattern matching, that is, the variation in the distance measurement calculation result can be reduced and the distance measurement accuracy can be improved, and a sharp surveillance image can be obtained at the same time.

The block diagram for demonstrating the structural example of the stereo image processing apparatus which concerns on this invention. Block diagram for explaining a configuration example of a peripheral resolution correction circuit The characteristic view which shows gain control of the correction signal in Example 1 of this invention The flowchart which shows distance measurement evaluation and the gain adjustment of a correction signal in Example 1 of this invention. Flowchart of stereo image processing method in Embodiment 2 of the present invention Flowchart showing distance measurement evaluation and gain adjustment of a correction signal in Embodiment 3 of the present invention Block diagram showing the configuration of a conventional stereo camera Illustration for explaining the problems of the prior art by specific examples

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. In the following description, a case where two sets of video signal output units are provided will be described, but the present invention can be similarly applied to a case where there are three or more sets of video signal output units.

  FIG. 1 is a block diagram for explaining a configuration example of a stereo image processing apparatus according to the present invention. This stereo image processing apparatus includes two sets of video signal output units including photographing lenses 101 and 102 and image sensors 103 and 104, a signal processing circuit 105, a peripheral resolution correction circuit 106, and a distance measurement circuit 107. Yes.

  As described above, a plurality of video signal output units are provided, and each video signal output unit includes a lens and an image sensor that forms a pair with the lens. The lenses of the plurality of video signal output units are arranged such that the same subject enters the angle of view and the optical axes are not parallel to each other. In this figure, the lenses 101 and 102 of the video signal output unit are arranged so that the same subject falls within the angle of view, and are arranged with an angle θ shifted from the two optical axes OA1 and OA2 that are parallel to each other. Yes.

  An image photographed by the video signal output unit is output to the signal processing circuit 105 as a video signal. The signal processing circuit 105 performs signal processing such as amplification and A / D conversion on this video signal, and outputs it to the peripheral resolution correction circuit 106.

  The peripheral resolution correction circuit 106 is provided in association with each of a plurality of video signal output units, and is a means for correcting and sharpening a video signal that is an output signal from the video signal output unit. is there.

  The distance measuring circuit 107 is a distance measuring unit that calculates distance values by calculating the distance between each pixel in the imaging and the subject based on the plurality of video signals corrected by each of the video signal correcting units described above. is there.

  FIG. 2 is a block diagram for explaining a configuration example of the peripheral resolution correction circuit 106. Since the stereo image processing apparatus shown in FIG. 1 includes two sets of video signal output units, two sets of peripheral resolution correction circuits are provided corresponding to these video signal output units. Since the configuration of the image correction circuit is the same, only the configuration of one peripheral resolution correction circuit will be described, and the description of the configuration of the other peripheral resolution correction circuit will be omitted.

  The peripheral resolution correction circuit 106 includes a correction signal generation circuit 201, a distance measurement evaluation circuit 202, a distance calculation circuit 203, a signal adjustment circuit 204, and an addition circuit 205.

  The correction signal generation circuit 201 is a means for generating a high-frequency signal for correction for the video signal output from the video signal output unit.

  The distance measurement evaluation circuit 202 receives the calculation result of the distance measurement circuit 107 described above, evaluates the variation of the distance measurement value from the distribution of distance measurement values of each pixel included in a specific area in the imaging, Ranging value variation evaluating means for evaluating variation of ranging values from the distribution of ranging values of specific pixels in the imaging in the frame. Here, the evaluation of variation is a variation degree of distance measurement values, for example, a variance value.

  The signal adjustment circuit 204 is an amplitude adjustment unit that adjusts the amplitude of the correction high-frequency signal so that the variation in the distance measurement value described above becomes small. The correction high-frequency signal after amplitude adjustment is returned to the main line by the addition circuit 205. This is added to adjust the sharpness of the photographed image.

  The distance calculation circuit 203 is a distance calculation unit that calculates how far each pixel is from the optical axis of the lens. When the distance calculation circuit 203 is used, the amplitude adjustment of the correction high-frequency signal described above is performed based on the distance calculated by the distance calculation circuit 203 in addition to the degree of variation in the distance measurement value. Can do.

  The procedure of processing executed in the present invention will be described below with reference to an embodiment. Note that the content of the processing is a method for processing a stereo image, and it is obvious that each processing (step) can be executed using a computer. Further, the program includes a plurality of video signal output units including a lens and an image sensor that forms a pair with the lens, and the lenses of the plurality of video signal output units have the same subject within the angle of view. A plurality of video signal output units whose optical axes are arranged non-parallel to each other and the video signal output units provided in association with each of the plurality of video signal output units are corrected to video signals that are output signals from the video signal output unit Is a program for processing a stereo image using a stereo image processing apparatus including a plurality of video signal correction units for sharpening by adding a stereo image for causing a computer to execute the following processes It is a processing program.

  FIG. 3 is a characteristic diagram showing gain control of the correction signal in the first embodiment of the present invention.

  FIG. 4 is a flowchart showing distance measurement evaluation and gain adjustment of a correction signal in Embodiment 1 of the present invention.

  The distance calculation circuit 203 obtains the distance from the optical axis center of the processing pixel or the square value of the distance, and this value is the horizontal axis (distance value) of the characteristic diagram shown in FIG.

  Here, the distance value r is expressed by the following equation (1) where X0 and Y0 are the horizontal and vertical positions of the pixel serving as the center of the optical axis, and X and Y are the horizontal and vertical positions of the pixel to be corrected. ) Or formula (2).

  As shown in FIG. 3, the control is performed so that the gain (vertical axis) of the correction signal increases as the distance value increases.

  In accordance with this characteristic, the signal adjustment circuit 204 increases the amplitude of the high-frequency signal for correction (G1 is increased) in accordance with the increase of the distance value from the optical axis center of the processing pixel, and the lens peripheral portion. Improve the sharpness of the image.

  On the other hand, the distance measurement evaluation circuit 202 executes the processing shown in FIG. First, in order to evaluate the variation of the distance calculation result of each pixel included in the specific area in the imaging among the distance calculation results sequentially output from the distance measuring circuit 107, the distance dispersion in the specific area The value Vt is calculated (S401).

  Here, it is assumed that the above-described specific area is, for example, a rectangular area having diagonally the horizontal and vertical positions (Xa, Ya) and (Xb, Yb) of the pixels. If the distance measurement value calculated by the distance measurement circuit 107 corresponding to each pixel position in the rectangular area is D (X, Y), the number of pixels in the area, the average value of the distance measurement values in the area, And a distance dispersion value becomes a following formula (3), a formula (4), and a formula (5), respectively.

  Next, the correction gain value G2 in the signal adjustment circuit 204 is increased by Δg + (S402), and the distance dispersion value Vt + 1 of the distance measurement value reflecting the increased gain is calculated by Expressions 4 and 5 (S403). The distance variance values are compared before and after the gain increase (S404), and if the difference is smaller than a certain threshold value TH (S404: Yes), the process returns to S401 so as to increase the correction gain again.

  On the other hand, if the distance variance value is greater than or equal to a certain threshold TH (S404: No), it is determined that the degree of variation in the distance measurement value has increased, the distance variance value Vt is calculated again (S405), and the correction gain G2 is set to Δg. The distance dispersion value Vt + 1 after the decrease of the correction gain G2 is obtained from the expressions (4) and (5) (S407), and the process returns to step S404 to evaluate the degree of variation of the distance measurement values again. To do.

  According to the above procedure, the correction gain G2 is controlled so that the variation of the distance measurement value is within a certain range, and the correction signal is adjusted by multiplying the high frequency signal by G2. In this way, control is performed to correct the sharpness of the contour imaged at the lens periphery, and the image is made similar to the density gradient and image level of the imaged image near the center of one lens. Thereby, the variation of the corresponding point position of pattern matching is reduced, and ranging accuracy can be improved.

  FIG. 5 is a flowchart of the stereo image processing method according to the second embodiment of the present invention.

  First, a high frequency signal for correction for sharpening the video signal output from the image sensor is generated (S501), and the distance value from the optical axis center described above is calculated by Expression (1) or Expression (2). (S502)

  Next, the correction gain G1 having the characteristics shown in FIG. 3 is obtained based on the distance value (S503). Then, the product of the correction gain G1 and the correction gain G2 obtained by the procedure described with reference to FIG. 4 is obtained as a total correction gain (S504).

  Subsequently, the correction high-frequency signal generated in step S501 is multiplied by the total correction gain (S505), and the correction-adjusted high-frequency signal is added to the main line signal (S506). By such a procedure, it is possible to sharpen the image around the lens to obtain an image with good image quality, and to adjust the sharpness to improve the distance measurement accuracy.

  FIG. 6 is a flowchart showing distance measurement evaluation and gain adjustment of a correction signal in the third embodiment of the present invention.

Of the ranging calculation results sequentially output from the ranging circuit 107, in order to evaluate the variation in the ranging calculation at a certain spatial position (i, j) (horizontal position i and vertical position j) on the image, N frames The distance dispersion value V _N is calculated (S601).

Assuming that the distance value of the spatial position at time t calculated by the distance measuring circuit 107 is D _t (i, j), the average value and the distance variance value of the distance values of N frames are respectively expressed by the following formulas (6 ) And Equation (7).

Next, the correction gain value G2 in the signal adjustment circuit 204 is increased by Δg + (S602), and the distance variance value V ′ _N (I, j) of the distance measurement value reflecting the increased gain is expressed by the above equation (6). ) And Equation (7) (S603).

  Subsequently, the distance variance values are compared before and after the gain increase (S604). If this difference is smaller than a certain threshold value TH (S604: Yes), the process returns to step S601 to increase the correction gain again.

On the other hand, if the distance variance value is greater than or equal to a certain threshold value TH (S604: No), it is determined that the variation degree of the distance measurement value has increased, and the distance variance value V _N (I, j) is calculated again (S605). Then, the correction gain G2 is decreased by Δg− (S606), and the distance dispersion value V ′ _N (I, j) after the decrease of the correction gain G2 is obtained from the expressions (6) and (7) (S607), step Returning to S604, the degree of variation in distance measurement values is evaluated again.

  In this way, the correction gain G2 is controlled so that the variation of the distance measurement value is within a certain range, and the correction signal is adjusted by multiplying the high frequency signal by G2. By correcting the video signal by such a procedure, the ranging accuracy can be improved. In this way, it is possible to improve the ranging accuracy with high spatial resolution.

  In the above embodiment, the correction high-frequency signal is described as a method of generating a high-frequency signal by applying a high-pass filter to the video signal output from the image pickup device. The method for generating the high-frequency signal is not limited. Moreover, although the example which calculates a dispersion value was shown for evaluation of a dispersion | variation degree, the correction control of this invention is realizable also by the evaluation method using another evaluation scale. Further, the arrangement angles of the photographing lenses 101 and 102 may not be the same θ.

  Each procedure described above can be executed by a computer by a program. That is, the present invention covers a stereo image processing program, and this program is a program for processing a stereo image using the stereo image processing apparatus according to the present invention.

  The stereo image processing apparatus according to the present invention is a stereo that can sharpen an image from the lens optical axis to the periphery, obtain a monitoring image with good image quality, and improve distance measurement accuracy with little variation in distance measurement values. This is useful as an image processing apparatus and a stereo image processing method.

101, 102 Imaging lens 103, 104 Image sensor 105 Signal processing circuit 106 Peripheral resolution correction circuit 107 Distance measurement circuit 201 Correction signal generation circuit 202 Distance measurement evaluation circuit 203 Distance calculation circuit 204 Signal adjustment circuit 205 Addition circuit 1001, 1002 Imaging lens 1003, 1004 Image sensor 1005 Image signal input circuit 1006 Image processing circuit 1007 Focus evaluation circuit 1008 System controller 1009 Motor driver 1010, 1011 Focus lens

Claims (12)

A plurality of video signal output units including a lens and an image sensor paired with the lens;
A plurality of video signal correction units provided in association with each of the plurality of video signal output units, for correcting and sharpening a video signal that is an output signal from the video signal output unit;
A distance measuring unit that calculates distance values by calculating the distance between each pixel in the imaging and the subject based on a plurality of video signals corrected by each of the video signal correcting units,
The lenses of the plurality of video signal output units have the same subject within the angle of view and the optical axes are arranged non-parallel to each other,
Any of the video signal correction units,
A correction high-frequency signal generation unit that generates a correction high-frequency signal for the video signal output from the video signal output unit;
In response to the calculation result of the ranging unit, a ranging value variation evaluating unit that evaluates the variation of the ranging value from the distribution of ranging values of each pixel included in the specific area in the imaging;
An amplitude adjustment unit that adjusts the amplitude of the high-frequency signal for correction so that variation in the distance measurement value is reduced;
Stereo image processing apparatus comprising: A plurality of video signal output units including a lens and an image sensor paired with the lens;
A plurality of video signal correction units provided in association with each of the plurality of video signal output units, for correcting and sharpening a video signal that is an output signal from the video signal output unit;
A distance measuring unit that calculates distance values by calculating the distance between each pixel in the imaging and the subject based on a plurality of video signals corrected by each of the video signal correcting units,
The lenses of the plurality of video signal output units have the same subject within the angle of view and the optical axes are arranged non-parallel to each other,
Any of the video signal correction units,
A correction high-frequency signal generation unit that generates a correction high-frequency signal for the video signal output from the video signal output unit;
In response to the calculation result of the distance measuring unit, a distance value variation evaluating unit that evaluates the variation of the distance value from the distribution of distance values of specific pixels in the imaging in a plurality of frames;
An amplitude adjustment unit that adjusts the amplitude of the high-frequency signal for correction so that variation in the distance measurement value is reduced;
Stereo image processing apparatus comprising: A plurality of video signal output units including a lens and an image sensor paired with the lens;
A plurality of video signal correction units provided in association with each of the plurality of video signal output units, for correcting and sharpening a video signal that is an output signal from the video signal output unit;
A distance measuring unit that calculates distance values by calculating the distance between each pixel in the imaging and the subject based on a plurality of video signals corrected by each of the video signal correcting units,
The lenses of the plurality of video signal output units have the same subject within the angle of view and the optical axes are arranged non-parallel to each other,
Any of the video signal correction units,
A correction high-frequency signal generation unit that generates a correction high-frequency signal for the video signal output from the video signal output unit;
In response to the calculation result of the ranging unit, a ranging value variation evaluating unit that evaluates the variation of the ranging value from the distribution of ranging values of each pixel included in the specific area in the imaging;
A distance calculator that calculates the distance of each pixel from the optical axis of the lens;
An amplitude adjustment unit that adjusts the amplitude of the high-frequency signal for correction based on the distance calculated by the distance calculation unit and the variation of the distance measurement value;
Stereo image processing apparatus comprising: A plurality of video signal output units including a lens and an image sensor paired with the lens;
A plurality of video signal correction units provided in association with each of the plurality of video signal output units, for correcting and sharpening a video signal that is an output signal from the video signal output unit;
A distance measuring unit that calculates distance values by calculating the distance between each pixel in the imaging and the subject based on a plurality of video signals corrected by each of the video signal correcting units,
The lenses of the plurality of video signal output units have the same subject within the angle of view and the optical axes are arranged non-parallel to each other,
Any of the video signal correction units,
A correction high-frequency signal generation unit that generates a correction high-frequency signal for the video signal output from the video signal output unit;
In response to the calculation result of the distance measuring unit, a distance value variation evaluating unit that evaluates the variation of the distance value from the distribution of distance values of specific pixels in the imaging in a plurality of frames;
A distance calculator that calculates the distance of each pixel from the optical axis of the lens;
An amplitude adjustment unit that adjusts the amplitude of the high-frequency signal for correction based on the distance calculated by the distance calculation unit and the variation of the distance measurement value;
Stereo image processing apparatus comprising: A plurality of video signal output units including a lens and an image sensor paired with the lens, wherein the lenses of the plurality of video signal output units have the same subject within an angle of view and the optical axes are not parallel to each other; A plurality of video signal output units disposed in the video signal output unit, and a plurality of video signal output units are provided in association with each other, and the video signal that is an output signal from the video signal output unit is corrected and sharpened. A method for processing a stereo image using a stereo image processing apparatus including a plurality of video signal correction units for performing the following steps, comprising the following steps.
A distance measuring step for calculating a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of image signals corrected by each of the image signal correction units:
A correcting high-frequency signal generating step for generating a correcting high-frequency signal for the video signal output from the video signal output unit:
Ranging value variation evaluation step of receiving the calculation result of the ranging step and evaluating the variation of the ranging value from the distribution of the ranging value of each pixel included in the specific area in the imaging:
An amplitude adjustment step of adjusting the amplitude of the correction high-frequency signal so that the variation of the distance measurement value is reduced: A plurality of video signal output units including a lens and an image sensor paired with the lens, wherein the lenses of the plurality of video signal output units have the same subject within an angle of view and the optical axes are not parallel to each other; A plurality of video signal output units disposed in the video signal output unit, and a plurality of video signal output units are provided in association with each other, and the video signal that is an output signal from the video signal output unit is corrected and sharpened. A method for processing a stereo image using a stereo image processing apparatus including a plurality of video signal correction units for performing the following steps, comprising the following steps.
A distance measuring step for calculating a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of image signals corrected by each of the image signal correction units:
A correcting high-frequency signal generating step for generating a correcting high-frequency signal for the video signal output from the video signal output unit:
Ranging value variation evaluating step for receiving the calculation result of the ranging step and evaluating the variation of the ranging value from the distribution of ranging values of specific pixels in the imaging in a plurality of frames:
An amplitude adjustment step of adjusting the amplitude of the correction high-frequency signal so that the variation of the distance measurement value is reduced: A plurality of video signal output units including a lens and an image sensor paired with the lens, wherein the lenses of the plurality of video signal output units have the same subject within an angle of view and the optical axes are not parallel to each other; A plurality of video signal output units disposed in the video signal output unit, and a plurality of video signal output units are provided in association with each other, and the video signal that is an output signal from the video signal output unit is corrected and sharpened. A method for processing a stereo image using a stereo image processing apparatus including a plurality of video signal correction units for performing the following steps, comprising the following steps.
A distance measuring step for calculating a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of image signals corrected by each of the image signal correction units:
A correcting high-frequency signal generating step for generating a correcting high-frequency signal for the video signal output from the video signal output unit:
Ranging value variation evaluation step of receiving the calculation result of the ranging step and evaluating the variation of the ranging value from the distribution of the ranging value of each pixel included in the specific area in the imaging:
A distance calculating step for calculating the distance of each pixel from the optical axis of the lens:
An amplitude adjustment step of adjusting the amplitude of the correction high-frequency signal based on the distance calculated in the distance calculation step and the variation of the distance measurement value: A plurality of video signal output units including a lens and an image sensor paired with the lens, wherein the lenses of the plurality of video signal output units have the same subject within an angle of view and the optical axes are not parallel to each other; A plurality of video signal output units disposed in the video signal output unit, and a plurality of video signal output units are provided in association with each other, and the video signal that is an output signal from the video signal output unit is corrected and sharpened. A method for processing a stereo image using a stereo image processing apparatus including a plurality of video signal correction units for performing the following steps, comprising the following steps.
A distance measuring step for calculating a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of image signals corrected by each of the image signal correction units:
A correcting high-frequency signal generating step for generating a correcting high-frequency signal for the video signal output from the video signal output unit:
Ranging value variation evaluating step for receiving the calculation result of the ranging step and evaluating the variation of the ranging value from the distribution of ranging values of specific pixels in the imaging in a plurality of frames:
A distance calculating step for calculating the distance of each pixel from the optical axis of the lens:
An amplitude adjustment step of adjusting the amplitude of the correction high-frequency signal based on the distance calculated in the distance calculation step and the variation of the distance measurement value: A plurality of video signal output units including a lens and an image sensor paired with the lens, wherein the lenses of the plurality of video signal output units have the same subject within an angle of view and the optical axes are not parallel to each other; A plurality of video signal output units disposed in the video signal output unit, and a plurality of video signal output units are provided in association with each other, and the video signal that is an output signal from the video signal output unit is corrected and sharpened. A stereo image processing program for processing a stereo image using a stereo image processing apparatus having a plurality of video signal correction units for causing the computer to execute the following steps.
A distance measuring step for calculating a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of image signals corrected by each of the image signal correction units:
A correcting high-frequency signal generating step for generating a correcting high-frequency signal for the video signal output from the video signal output unit:
Ranging value variation evaluation step of receiving the calculation result of the ranging step and evaluating the variation of the ranging value from the distribution of the ranging value of each pixel included in the specific area in the imaging:
An amplitude adjustment step of adjusting the amplitude of the correction high-frequency signal so that the variation of the distance measurement value is reduced: A plurality of video signal output units including a lens and an image sensor paired with the lens, wherein the lenses of the plurality of video signal output units have the same subject within an angle of view and the optical axes are not parallel to each other; A plurality of video signal output units disposed in the video signal output unit, and a plurality of video signal output units are provided in association with each other, and the video signal that is an output signal from the video signal output unit is corrected and sharpened. A stereo image processing program for processing a stereo image using a stereo image processing apparatus having a plurality of video signal correction units for causing the computer to execute the following steps.
A distance measuring step for calculating a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of image signals corrected by each of the image signal correction units:
A correcting high-frequency signal generating step for generating a correcting high-frequency signal for the video signal output from the video signal output unit:
Ranging value variation evaluating step for receiving the calculation result of the ranging step and evaluating the variation of the ranging value from the distribution of ranging values of specific pixels in the imaging in a plurality of frames:
An amplitude adjustment step of adjusting the amplitude of the correction high-frequency signal so that the variation of the distance measurement value is reduced: A plurality of video signal output units including a lens and an image sensor paired with the lens, wherein the lenses of the plurality of video signal output units have the same subject within an angle of view and the optical axes are not parallel to each other; A plurality of video signal output units disposed in the video signal output unit, and a plurality of video signal output units are provided in association with each other, and the video signal that is an output signal from the video signal output unit is corrected and sharpened. A stereo image processing program for processing a stereo image using a stereo image processing apparatus having a plurality of video signal correction units for causing the computer to execute the following steps.
A distance measuring step for calculating a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of image signals corrected by each of the image signal correction units:
A correcting high-frequency signal generating step for generating a correcting high-frequency signal for the video signal output from the video signal output unit:
Ranging value variation evaluation step of receiving the calculation result of the ranging step and evaluating the variation of the ranging value from the distribution of the ranging value of each pixel included in the specific area in the imaging:
A distance calculating step for calculating the distance of each pixel from the optical axis of the lens:
An amplitude adjustment step of adjusting the amplitude of the correction high-frequency signal based on the distance calculated in the distance calculation step and the variation of the distance measurement value: A plurality of video signal output units including a lens and an image sensor paired with the lens, wherein the lenses of the plurality of video signal output units have the same subject within an angle of view and the optical axes are not parallel to each other; A plurality of video signal output units disposed in the video signal output unit, and a plurality of video signal output units are provided in association with each other, and the video signal that is an output signal from the video signal output unit is corrected and sharpened. A stereo image processing program for processing a stereo image using a stereo image processing apparatus having a plurality of video signal correction units for causing the computer to execute the following steps.
A distance measuring step for calculating a distance value by calculating a distance between each pixel in the image pickup and a subject based on the plurality of image signals corrected by each of the image signal correction units:
A correcting high-frequency signal generating step for generating a correcting high-frequency signal for the video signal output from the video signal output unit:
Ranging value variation evaluating step for receiving the calculation result of the ranging step and evaluating the variation of the ranging value from the distribution of ranging values of specific pixels in the imaging in a plurality of frames:
A distance calculating step for calculating the distance of each pixel from the optical axis of the lens:
An amplitude adjustment step of adjusting the amplitude of the correction high-frequency signal based on the distance calculated in the distance calculation step and the variation of the distance measurement value:

Images