JP5146959B2 - Parallax sensor and parallax image generation method - Google Patents

Description

  The present invention relates to a technique for generating a parallax image that generates a parallax image from a stereo image, and particularly to a technique for generating a parallax image that can be realized at high speed, with a small circuit scale, and with low power consumption.

The present invention relates to an improvement technique of the invention described in Patent Document 1 previously proposed by the present inventor and applied for a patent.
A parallax sensor as a conventional technique will be described below.

  With the spread of CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor) image sensors, information devices can easily handle image information. Recently, most mobile phones have built-in small cameras to facilitate image data communication. Many automobiles are equipped with an image sensor to cover the blind spots from the driver's seat to help drive safely. However, what is obtained from a conventional image sensor is merely two-dimensional information, and three-dimensional information including distance (depth) information cannot be obtained. Therefore, a new image sensor capable of obtaining distance information is demanded.

  There are mainly two types of high-speed detection methods for the distance to the object. One is an active ranging method in which electromagnetic waves such as highly directional laser light and radio waves are irradiated onto an object and the distance from the object is measured with a time delay of the reflected signal. The other is a passive ranging method in which the correlation between images is calculated using two image sensors and the distance is calculated by extracting the parallax between the two sensors.

  Aircraft, ships, and the like are equipped with a radar, which is a representative example of an active distance measuring method, and can monitor the distance to objects around it and enable safe navigation. Recently, millimeter-wave radar has also been installed in automobiles to monitor the distance between vehicles and help prevent collisions. Thus, at present, radar that is an active distance measuring method is used for detecting a high-speed distance.

  However, since the active ranging method uses reflected signals of electromagnetic waves such as laser light and radio waves, depending on the object to be detected, irregular reflection may occur, which may interfere with accurate distance detection due to signal interference, There is a problem in that a reflected signal is not sufficiently obtained due to absorption and there is a possibility that the object cannot be detected reliably.

  On the other hand, since the passive distance measurement method does not irradiate electromagnetic waves, there is an advantage that the problem of signal interference and insufficient reflected signals in the active distance measurement method can be avoided.

  However, since this passive distance measurement method requires a large amount of calculation for the correlation processing of two pieces of image data, it has been difficult for the conventional correlation processing LSI to perform high-speed processing that can cope with high-speed movement. For this reason, it has not been put into practical use.

FIG. 7 is a diagram for explaining the principle of distance detection by binocular parallax. As shown in FIG. 7A, an image pickup device corresponding to the right eye and an image pickup device corresponding to the left eye are set apart from each other and imaged. Here, an image captured by an image sensor corresponding to the left eye (hereinafter referred to as “left eye image”) is represented by {a ^(L) _{i, j} | i = 1, 2,..., N, j = 1, 2, ..., m}. An image captured by an image sensor corresponding to the right eye (hereinafter referred to as a “right eye image”) {a ^(R) _{i, j} | i = 1, 2,..., N, j = 1, 2,. , M}.

With respect to the same object, the positions of the objects imaged by the left and right imaging elements are shifted according to the distance from the imaging element to the object. Therefore, when considering only the horizontal correlation between the captured left-eye image and right-eye image, the correlation between all (a ^(L) _{i, k} , a ^(R) _{j, k} ) pairs is calculated. Then, the distance of each object can be detected by the coordinates represented by the set having the greatest correlation.

  In FIG. 7A, the three circles A, B, and C indicate the object. When these objects are imaged by the left and right imaging elements to create a correlation matrix between the left eye image and the right eye image, the result is as shown in FIG. In FIG. 7B, it is assumed that there is a correlation function at the position of the intersection of the lines 1 to n of the left eye image and the lines 1 to n of the right eye image. A large correlation is detected with respect to the objects A, B, and C in FIG. 7A at the three circle positions A, B, and C shown in FIG. Therefore, if a coordinate having a large correlation is detected on the correlation matrix, and the coordinate is converted into an oblique coordinate in FIG. 7A, the distance to the object can be detected. The oblique coordinates shown in FIG. 7A can be determined by the positions of the left and right imaging elements and their relative angles. Therefore, the coordinate conversion from the correlation matrix to the oblique coordinates can be easily performed by referring to a conversion table prepared in advance.

  FIG. 8 is a diagram showing a configuration example of the parallax sensor LSI described in Patent Document 1 previously proposed. The parallax sensor 1 includes two image sensors 2a and 2b, a sequencer 3, two voltage / pulse width conversion circuit arrays 4a and 4b, two pulse signal comparison circuit arrays 5a and 5b, a correlation detection circuit matrix 6, and a sequencer 7. I have.

  The left and right imaging elements 2a and 2b play the role of eyes. Hereinafter, for the sake of convenience, the side of the image sensor 2a is referred to as the left eye, and the side of the image sensor 2b is referred to as the right eye. The imaging elements 2a and 2b convert light incident on the imaging surface into a voltage signal and output the voltage signal. Here, it is assumed that an image sensor such as a CCD light receiving element is used as the imaging elements 2a and 2b. The sequencer 3 outputs a readout signal for line selection to the image sensors 2a and 2b.

  The voltage / pulse width conversion circuit arrays 4a and 4b convert analog voltage signals (hereinafter referred to as “pixel signals”) of the pixels output in parallel from the image pickup devices 2a and 2b into pulse widths in parallel. And output as a pulse width pixel signal.

  The pulse signal comparison circuit arrays 5a and 5b compare the pulse width pixel signals inputted in parallel with each other and output them as comparison pulse signals.

  The correlation detection circuit matrix 6 applies all the combinations of the comparison pulse signal output from the pulse signal comparison circuit array 5a on the left eye side and the comparison pulse signal output from the pulse signal comparison circuit array 5b on the right eye side. Perform correlation calculation and output as correlation signal. The sequencer 7 outputs an output timing control signal for controlling the output timing of the correlation signal of the correlation detection circuit matrix 6 to the correlation detection circuit matrix 6. The correlation detection circuit matrix 6 sequentially outputs correlation signals according to the output timing control signal.

  FIG. 9 is a diagram showing a more detailed circuit configuration of the parallax sensor of FIG. In FIG. 9, the same parts as those in FIG.

  The imaging devices 2a and 2b have a matrix (pixel matrix) of pixels of m rows in the vertical direction and n rows in the horizontal direction. The image sensors 2a and 2b are selected in the same row by the sequencer 3, and pixel signals of n pixels in the row are output in parallel to the voltage / pulse width conversion circuit arrays 4a and 4b.

  Each of the voltage / pulse width conversion circuit arrays 4a and 4b has a configuration in which n voltage / pulse width conversion circuits 8 are arranged in parallel. Each voltage / pulse width conversion circuit 8 receives pixel signals output from the image sensors 2a and 2b, respectively. A common ramp voltage is input from the synchronization control circuit 13 (see FIG. 10) to all the voltage / pulse width conversion circuits 8 of the voltage / pulse width conversion circuit arrays 4a and 4b. Thereby, all the voltage / pulse width conversion circuits 8 can convert the pixel signal into the pulse width pixel signal at the same timing.

  Each of the pulse signal comparison circuit arrays 5a and 5b has a configuration in which n-1 pulse signal comparison circuits 9 are arranged in parallel. Each pulse signal comparison circuit 9 receives a pulse width pixel signal output from two adjacent voltage / pulse width conversion circuits 8. Each pulse signal comparison circuit 9 compares the two input pulse width pixel signals, and outputs the difference between the two pulses in the + direction and the difference in the-direction as a comparison pulse signal.

  The correlation detection circuit matrix 6 has a configuration in which (n−1) × (n−1) correlation detection circuits 10 are arranged in a diamond shape of (n−1) rows (n−1) columns. Here, for the sake of convenience, the hypotenuse to which the comparison pulse signal output from the left eye side pulse signal comparison circuit array 5a is input is called the left hypotenuse, and the comparison pulse signal output from the right eye side pulse signal comparison circuit array 5b is The input hypotenuse is called the right hypotenuse.

  The comparison pulse signal output from the i-th pulse signal comparison circuit 9 in the pulse signal comparison circuit array 5a on the left eye side is input to the correlation detection circuit 10 belonging to the i-th from the top along the left oblique side. Comparison pulse signals output from the pulse signal comparison circuit 9 in the pulse signal comparison circuit array 5a on the left eye side are commonly input to the correlation detection circuits 10 arranged in parallel with the right oblique side.

  The comparison pulse signal output from the jth pulse signal comparison circuit 9 in the pulse signal comparison circuit array 5b on the right eye side is input to the correlation detection circuit 10 belonging to the jth from the top along the right oblique side. Comparison pulse signals output from the pulse signal comparison circuit 9 in the pulse signal comparison circuit array 5b on the right eye side are commonly input to the correlation detection circuits 10 arranged in parallel with the left oblique side.

A common bias voltage V _b and a reset signal Reset are given to all the correlation detection circuits 10.

  A common readout line is connected to each column in the correlation detection circuits 10 arranged in the vertical direction. A common read signal Read is input from the sequencer 7 via this read line. Further, a common output line is connected for each row to the correlation detection circuits 10 arranged in the horizontal direction. Each correlation detection circuit 10 converts the pulse width of the input comparison pulse signal into a current value and outputs it as a correlation signal to the output line. A current-voltage conversion circuit 11 such as a current mirror circuit is connected to the end of each output line. The current value of the correlation signal output from each correlation detection circuit 10 is converted into a voltage value and output to an external circuit.

  The sequencer 7 includes a shift register 12. When a read signal is input to the left shift register 12, the read signal moves to the right shift register 12 every clock. Accordingly, correlation signals are sequentially read from the correlation detection circuits 10 belonging to the left column.

  Next, details of the voltage / pulse width conversion circuit 8 in FIG. 9 will be described. FIG. 10 is a diagram showing the configuration of the voltage / pulse width conversion circuit 8. The voltage / pulse width conversion circuit 8 in this embodiment is configured by a logic threshold variable modulation inverter circuit (VT-INV).

  The logic threshold adjustable inverter circuit (VT-INV) includes a CMOS type inverter capable of modulating a gain coefficient by a gain coefficient control voltage applied to a control gate, and the control gate controls the gain coefficient. It consists of the structure connected to the terminal (CNT). The gain coefficient control terminal (CNT) in the voltage / pulse width conversion circuit 8 is connected to the synchronization control circuit 13. The synchronization control circuit 13 includes a ramp signal generation circuit. The ramp voltage generated by the ramp signal generation circuit is commonly input to the gain coefficient control terminal (CNT) of all the voltage / pulse width conversion circuits 8. Accordingly, all the voltage / pulse width conversion circuits 8 perform voltage / pulse width conversion at the same timing.

FIG. 11 is a diagram showing an operation example of the voltage / pulse width conversion circuit 8. The ramp voltage (Ramp Sig.) Output from the synchronization control circuit 13 has a sawtooth shape as shown in the uppermost stage of FIG. When this ramp voltage is input to the gain coefficient control terminal (CNT), the logical threshold voltage V _inv of the voltage / pulse width conversion circuit 8 changes as shown by the dotted line in FIG. That is, as the ramp voltage increases, the logical threshold voltage V _inv decreases. When the logical threshold voltage V _inv becomes smaller than the voltage (Analog V _in ) of the pixel signal, the pulse width pixel signal (OUT) output to the output terminal of the voltage / pulse width conversion circuit 8 becomes H level. . When the ramp voltage returns to the lowest level again, the logic threshold voltage V _inv becomes maximum and the pulse width pixel signal (OUT) becomes L level.

  As described above, the timing at which the pulse width pixel signal (OUT) becomes L level is constant because it is determined by the ramp voltage. However, the timing at which the pulse width pixel signal (OUT) becomes the H level is earlier as the voltage of the pixel signal is higher, and is delayed as the voltage of the pixel signal is lower. Therefore, the time during which the pulse width pixel signal (OUT) is at the H level (pulse width of the pulse width pixel signal) is proportional to the voltage of the pixel signal. That is, the voltage value of the pixel signal is converted into the pulse width of the pulse width pixel signal.

Next, the details of the pulse signal comparison circuit 9 in FIG. 9 will be described. FIG. 12 is a diagram showing the configuration of the pulse signal comparison circuit 9. The pulse signal comparison circuit 9 is composed of four inverters 41, 42, 45, 46 and two AND gates 43, 44. This circuit outputs the following values as output values (comparison pulse signals) OUT ₊ and OUT _{− to} the input terminals INa and INb.
OUT ₊ = INa∧ (/ INb)
_{OUT - = INb∧ (/ INa)} ······ ( number 1)
Here, (/ INb) and (/ INa) represent inverted signals of INb and INa, respectively, and ∧ represents a logical product.

  FIG. 13 is a time chart showing an operation example of the pulse signal comparison circuit 9. Outputs (pulse width pixel signals) of adjacent voltage / pulse width conversion circuits are input to the input terminals INa and INb, respectively. Since the end (falling) of the pulse of each input signal is determined by the falling edge of the lamp voltage, it is aligned at a certain time. On the other hand, the start (rise) of the pulse of each input signal changes in proportion to the magnitude of the pixel signal.

When the input signal of the input terminal INa is longer than the input signal of the input terminal INb, the input signal of the input terminal INa rises before the input signal of the input terminal INb. When INa = 1 and INb = 0, from Equation (1), OUT ₊ = 1 and OUT ₋ = 0. Further, when INa = 1 and INb = 1, OUT ₊ = 0 and OUT ₋ = 0 from (Equation 1). Therefore, a differential pulse of INa−INb is output to the comparison pulse signal OUT ₊ .

On the other hand, when the input signal of the input terminal INb is longer than the input signal of the input terminal INa, the input signal of the input terminal INb rises before the input signal of the input terminal INa. When INa = 0 and INb = 1, according to (Equation 1), OUT ₊ = 0 and OUT ₋ = 1. Further, when INa = 1 and INb = 1, OUT ₊ = 0 and OUT ₋ = 0 from (Equation 1). Therefore, a differential pulse of INb−INa is output to the comparison pulse signal OUT ₋ .

If the input signal of the input terminal INb and the input signal of the input terminal INa have the same length, no pulse is output to the comparison pulse signals OUT ₊ and OUT ₋ .

  In this manner, by mapping the voltage value of the pixel signal, which is an analog voltage signal, to the pulse width of the pulse width pixel signal, it is possible to perform pixel value difference calculation using a simple logic circuit.

  In this circuit, a shorter pulse is output as the correlation between the input signal at the input terminal INb and the input signal at the input terminal INa increases.

  Next, details of the correlation detection circuit 10 in FIG. 9 will be described. FIG. 14 is a diagram illustrating the configuration of the correlation detection circuit 10. The correlation detection circuit 10 includes a capacitor 50, current switch circuits 51 and 52, a current source 53, a reset switch 54, an output circuit 55, and a readout switch 56.

The capacitor 50 stores a charge for generating a correlation signal. The current switch circuit 51 is subjected to conduction / cut-off control according to the truth value of the exclusive OR of the input signals input from the input terminals R + and L +, and discharges the charge stored in the capacitor 50 at a constant current in the conduction state. The current switch circuit 52 is subjected to conduction / cut-off control according to the truth value of the exclusive OR of the input signals input from the input terminals R− and L−, and discharges the charge stored in the capacitor 50 at a constant current in the conduction state. Let The current source 53 is a circuit for allowing a constant discharge current to flow when the current switch circuits 51 and 52 are turned on. Reset switch 54 is rendered conductive when the reset signal (Reset) is input, and supplies the charge from the power source to the capacitor 50, the voltage across the capacitor 50 and the power supply voltage V _d.

  The output circuit 55 is a circuit for supplying a current proportional to the voltage of the capacitor 50, and is a circuit for converting the voltage of the capacitor 50 into a current and outputting the current. The output circuit 55 is configured by a MOS transistor. The voltage of the capacitor 50 is input to the gate and output as a drain current. Thus, the voltage of the capacitor 50 is constant during output if the leakage current can be ignored. Therefore, it is possible to output a stable correlation signal. The read switch 56 is for turning on / off the current output by the output circuit 55.

  In the correlation detection circuit 10, the degree of correlation of the input signal is expressed by the amount of charge accumulated in the capacitor 50. Immediately after resetting, the amount of accumulated charge is maximum. The lower the correlation level of the input signal is, the more electric charge is discharged, and the accumulated charge amount of the capacitor 50 is reduced. Thereby, correlation calculation is realized. The discharge current flows to the ground side through one of the current switch circuits 51 and 52.

FIG. 15 is a time chart showing an operation example of the correlation detection circuit 10. First, First, the reset signal (Reset) is a 0 (t1), the voltage Vc of the capacitor 50 is a power supply voltage V _d. Then, after setting the reset signal to 1 (t2), the comparison pulse signals OUT + and OUT− output from the pulse signal comparison circuit 9 on the left eye side are input to L + and L−, and the right eye side is input to R + and R−. Comparison pulse signals OUT + and OUT− output from the pulse signal comparison circuit 9 are input.

  When one of R + and L + is 1 and the other is 0 (t3 to t4, t5 to t6, t10 to t11, t12 to t13), the current switch circuit 51 becomes conductive. Accordingly, at this time, the electric charge of the capacitor 50 is discharged, and the voltage of the capacitor 50 decreases.

  When one of R- and L- is 1 and the other is 0 (t22 to t23, t24 to t25, t29 to t30, t31 to t32), the current switch circuit 52 becomes conductive. Accordingly, at this time, the electric charge of the capacitor 50 is discharged, and the voltage of the capacitor 50 decreases.

  When R + and L + are both 0 or 1, and R− and L− are both 0 or 1 (t1 to t3, t4 to t5, t6 to t10, t11 to t12, t13 to t22, t23 to t24, From t25 to t29, t30 to t31, and t32 to), the current switch circuits 51 and 52 are both cut off. Accordingly, at this time, the voltage Vc of the capacitor 50 is constant.

  After the ramp signal falls, the voltage Vc of the capacitor 50 is determined. When the correlation between R + and L + is small, or when the correlation between R− and L− is small, the final voltage Vc of the capacitor 50 becomes low. Conversely, when the correlation between R + and L + is large, or when the correlation between R− and L− is large, the final voltage Vc of the capacitor 50 is kept high.

  After the voltage Vc of the capacitor 50 is determined, the read signal (read) becomes 1 (t7, t14, t19, t26, t33), and the read switch 56 becomes conductive. As a result, the output circuit 55 outputs a current having a magnitude proportional to the voltage Vc of the capacitor 50.

  After the output is completed, the reset signal is set to 0 again (t8, t15, t20, t27), and the same correlation detection calculation is repeated.

  The overall operation of the parallax sensor according to the present embodiment configured as described above will be described below.

  FIG. 16 is a time chart showing an example of the operation of the parallax sensor 1. In FIG. 16, for convenience of explanation, the display is focused on a certain two pixels, but the same operation is performed in parallel on all the pixels.

  First, pixel signals a and b are output from the image sensors 2a and 2b (t0). As a result, the input voltage is determined in each voltage / pulse width conversion circuit 8 in the voltage / pulse width conversion circuit arrays 4a and 4b. In the example of FIG. 16, the pixel signal a has a higher value than the pixel signal b.

Then, the input pulse of the reset signal (Reset) (t1~t2), voltage V _c of the capacitor 50 is set to V _d relative to the correlation detection circuit 10.

Next, the synchronization control circuit 13 starts outputting the ramp signal (Ramp Sig.), And the voltage of the ramp signal gradually increases. Accordingly, the logical threshold voltage V _inv decreases in each voltage / pulse width conversion circuit 8. In the example of FIG. 16, since the voltage of the pixel signal a is higher than the voltage of the pixel signal b, first, in the voltage / pulse width conversion circuit 8 on the left eye side, the logical threshold voltage V _inv is equal to the pixel signal a. It becomes lower than the voltage (t3). As a result, the pulse width pixel signal (APWC OUT-a) output from the voltage / pulse width conversion circuit 8 on the left eye side becomes 1. At this time, the pulse width pixel signal (APWC OUT-b) output from the voltage / pulse width conversion circuit 8 on the right eye side is zero. Accordingly, the output (comparison pulse signal) DIFC OUT + of the pulse signal comparison circuit 9 becomes 1.

When the ramp signal further increases with the passage of time, the logical threshold voltage V _inv becomes lower than the voltage of the pixel signal b in the voltage / pulse width conversion circuit 8 on the right eye side (t4). As a result, the pulse width pixel signal (APWC OUT-b) output from the voltage / pulse width conversion circuit 8 on the right eye side becomes 1. At this time, the pulse width pixel signal (APWC OUT-a) output from the voltage / pulse width conversion circuit 8 on the left eye side is 1. Accordingly, the output (comparison pulse signal) DIFC OUT + of the pulse signal comparison circuit 9 becomes zero. The pulse width (t3 to t4) of the output (comparison pulse signal) DIFC OUT + of the pulse signal comparison circuit 9 represents the correlation between pixels.

On the other hand, while the output (comparison pulse signal) DIFC OUT + of the pulse signal comparison circuit 9 is 1 (t3 to t4), the current switch circuit 51 becomes conductive. Accordingly, during this time, the electric charge of the capacitor 50 is discharged to the ground via the switch circuit 51. Then, at the time (t4) when the comparison pulse signal DIFC OUT + falls, the voltage V _c of the capacitor 50 is determined. Thereafter, the ramp signal falls (t5), and all the correlation calculation processes are finished here.

  Next, the reading period starts. In the read period (t6), the clock CLK is supplied to the shift register 12 of the sequencer 7. Then, 1 is input as the input signal SRin to the leftmost shift register 12 for a certain period.

  The pulse width Ts of the input signal SRin is usually several times the width of the clock CLK. Since this pulse width Ts affects the size of the object that can be detected, it can be changed according to the situation. In general, the larger Ts is, the easier it is to recognize a large object and the fine noise is reduced. On the other hand, if Ts is reduced, a small object can be easily recognized, but the amount of noise increases. Therefore, the frequency characteristic of the high frequency filter can be set by setting Ts.

  The pulse of the input signal SRin moves from the left shift register 12 toward the right shift register 12 in accordance with the clock CLK. The output of the shift register 12 is input to the correlation detection circuit 10 of each column as a read signal (Read). Accordingly, the correlation signals held in the capacitors 50 in the correlation detection circuits 10 in each column of the correlation detection circuit matrix 6 are sequentially read from the left to the right.

  FIG. 17 is a diagram illustrating another configuration of the pulse signal comparison circuit of the parallax sensor described in Patent Document 1. In FIG. Other configurations are the same as those in FIG. 12, and a description thereof will be omitted.

  This pulse signal comparison circuit 9 ′ is different from the pulse signal comparison circuit 9 of FIG. 12 in that AND gates 47 and 48 are used instead of the inverters 41 and 42. In the AND gates 47 and 48, the input signals INa and INb are input to the input terminals on one side, and the selection signals Cna and Cnb are input to the input terminals on the other side. When Cna is set to 0, the input signal INb is output as it is to OUT−. When Cnb is 0, the input signal INa is output as it is to OUT +.

  As a result, in the pulse signal comparison circuit arrays 5a and 5b, the outputs of the voltage / pulse width conversion circuit arrays 4a and 4b can be directly input to the correlation detection circuit matrix 6 without comparing adjacent signals. Therefore, in this case, the correlation detection circuit matrix 6 can perform the correlation calculation process on the pixels of the left eye image and the right eye image as they are.

  Accordingly, it is possible to switch between direct correlation processing of pixel signals and correlation processing of image change signals by operating the selection signals Cna and Cnb according to the application.

JP 2005-265457 A

Since the parallax correlation data obtained by the above-described prior art has a characteristic pattern c where the object a exists, as shown in FIG. 18, by detecting the coordinates of the characteristic pattern c, The horizontal position and depth of the object (distance to the object) can be specified. However, since the band-like patterns b1 and b2 always appear in both oblique directions of the image at the position where the feature pattern of the object a exists, in order to detect the feature pattern c of the object a, a process such as pattern matching is performed. Is required.
Therefore, an object of the present invention is to provide a parallax sensor and a parallax image generation method capable of obtaining only a feature pattern of an object by deleting a belt-like pattern or the like.

In order to solve the above problems, a parallax sensor according to the present invention captures an object and outputs a first image as a pixel signal that is an analog voltage signal;
A second imaging element that images the object from an angle different from that of the first imaging element and outputs a second image as a pixel signal that is an analog voltage signal;
A plurality of voltage / pulse width conversion circuits for converting each of the pixel signals output from the first and second image sensors into a pulse width pixel signal having a pulse width having a length proportional to the voltage value of each pixel signal. When,
A synchronous control circuit that performs timing control so that all the voltage / pulse width conversion circuits simultaneously convert each pixel signal into a pulse width pixel signal in parallel;
A plurality of pulse signals that compare two pulse width pixel signals output from the adjacent voltage / pulse width conversion circuits and output a difference between two pulse width pixel signals in a positive direction and a negative direction as comparison pulse signals, respectively. A comparison circuit;
Two comparison pulse signals each composed of a combination of a comparison pulse signal that is an output of the pulse signal comparison circuit corresponding to the first image and a comparison pulse signal that is an output of the pulse signal comparison circuit corresponding to the second image In contrast, multiple correlation detections that convert the total pulse length of the differential pulse obtained by taking the exclusive OR of both into a signal of voltage value or current value proportional to the total pulse length and output this signal as a correlation signal Circuit,
In a parallax sensor comprising:
A zero signal detection circuit for detecting that the pulse width of the comparison pulse signal, which is an output of the pulse signal comparison circuit, is equal to or less than a certain length, and the comparison pulse signal or the predetermined signal according to the output signal of the zero signal detection circuit A zero correlation removal signal generation circuit comprising a selection circuit that selects the selected signal and outputs the selected signal to the correlation detection circuit.

In the method for generating a parallax image according to the present invention, the first imaging element captures an image of an object, and the first image is output as a pixel signal that is an analog voltage signal. A first step of imaging the object from an angle different from that of the image sensor and outputting a second image as a pixel signal that is an analog voltage signal;
Each pixel signal output from the first and second imaging elements is pulsed with a pulse width having a length proportional to the voltage value of the pixel signal simultaneously and in parallel by a plurality of voltage / pulse width conversion circuits. A second step of converting to a pixel signal;
A third step of comparing two pulse width pixel signals output from the adjacent voltage / pulse width conversion circuits, and outputting a difference between the two pulse width pixel signals in the positive direction and a negative direction as comparison pulse signals, respectively. When,
A fourth step of selecting and outputting the comparison pulse signal or a predetermined signal when it is detected that the pulse width of the comparison pulse signal is equal to or less than a certain length;
A comparison pulse signal corresponding to the first image is selected by a plurality of correlation detection circuits. However, when a predetermined signal is selected in the fourth step, the predetermined signal corresponds to the second image. The two exclusive pulse logics for two pulse width pixel signals each comprising a combination of comparison pulse signals (however, if a predetermined signal is selected in the fourth step). A fifth step of converting the total pulse length of the summed differential pulse into a signal having a voltage value or a current value proportional to the total pulse length and outputting this signal as a correlation signal;
It is characterized by having.

In the present invention, when the pulse width of the pulse signal given to the correlation circuit is shorter than a certain length to be regarded as a zero signal, that is, when adjacent pulse signals are substantially equal, by modulating the pulse signal, The correlation value other than the target is forcibly reduced. As a result, only the correlation pattern of the object remains. In order to realize this function, a circuit for detecting a pulse width shorter than a certain length and a circuit for forcibly making the pulse signal a predetermined pulse signal by the output signal are provided.
In the circuit configuration of the present invention, both the circuit area and the power consumption can be made smaller than before, so that it is possible to integrate two image sensor functions and their correlation processing circuits in one LSI chip.
Therefore, the parallax sensor of the present invention enables high-speed distance measurement, and can significantly reduce the device cost and power consumption compared to radar. Since this parallax sensor is a passive distance measuring method, there is an advantage that the problem of signal interference in the active method can be avoided.
The zero signal detection circuit has a function of inputting a comparison pulse signal via a MOSFET and adjusting a pulse width to be detected according to a voltage value applied to the gate of the MOSFET, whereby a plurality of APW conversion circuits and pulse signals are provided. Removal of whisker-like pulses caused by variations in characteristics of a large number of transistors constituting the comparison circuit can be facilitated.

  According to the present invention, when the pulse width of the pulse signal given to the correlation circuit is shorter than a certain length, the correlation signal is obtained by modulating the pulse signal and leaving only the correlation pattern of the object. Since only the part of the object has a characteristic value, the position can be detected only by simple threshold processing, and the calculation cost of post-processing can be greatly reduced. As a result, the cost of the apparatus can be reduced and the processing time can be shortened.

Hereinafter, embodiments of the present invention will be described with reference to FIGS.
1 is a diagram illustrating a configuration of a parallax sensor according to an embodiment of the present invention, FIG. 2 is a circuit diagram illustrating a configuration of a zero correlation removal signal generating circuit according to the present embodiment, and FIG. 3 is according to the present embodiment. 4 is a circuit diagram showing a configuration of a zero signal detection circuit, FIG. 4 is a circuit diagram showing a configuration of an RS flip-flop circuit according to the present embodiment, and FIG. 5 is a correlation in which a zero correlation removal function according to the present embodiment is introduced. FIG. 6 is a chart showing an example of correlation data according to the present embodiment.
As shown in FIG. 1, the parallax sensor circuit according to the present embodiment is provided with zero correlation removal signal generation circuit arrays 15a and 15b in the conventional parallax sensor circuit proposed in Patent Document 1 shown in FIG. Is. Since other configurations are the same as the conventional configuration, the same reference numerals are given and description thereof is omitted.

  (N-1) (n is the horizontal direction of the pixels constituting the left-eye and right-eye image pickup devices 2a and 2b) constituting the zero-correlation removal signal generation circuit arrays 15a and 15b, respectively, which is a feature of the present invention. FIG. 2 shows an example of the configuration of the zero correlation elimination signal generation circuit 16 of the number of rows). The zero correlation removal signal generation circuit 16 includes a zero signal detection circuit 161 and two selection circuits 162 and 163. The zero signal detection circuit 161 detects that the pulse widths of the comparison pulse signals OUT + and OUT− that are the outputs of the pulse signal comparison circuit 9 are equal to or less than a certain length, and the selection circuits 162 and 163 are zero. A comparison pulse signal or a predetermined signal (H / L) is selected in accordance with the output signal (SetOut) of the signal detection circuit 161 and is output to the next correlation detection circuit 10 as MOUT + and MOUT−. That is, the selection circuits 162 and 163 output the value given to the X terminal when the SetOut output given to the Sel terminal is 0, that is, the H / L which is a predetermined value, and when the SetOut output is 1. A value given to the Y terminal, that is, OUT + and OUT−, which are outputs of the pulse signal comparison circuit 9, is output.

As shown in FIG. 3, the zero signal detection circuit 161 includes a NOR circuit 1611, a NOT circuit 1612, a MOSFET 1613, an R-S flip-flop 1614, and a NAND circuit 1615. Note that the MOSFET 1613 has a function of adjusting a pulse width to be detected according to a voltage value applied to the gate thereof, and is generated due to variations in characteristics of a plurality of transistors or the like constituting a plurality of APW modulation circuits and pulse signal comparison circuits. Removal of whisker-like pulses can be facilitated.
A configuration example of the RS flip-flop 1614 is shown in FIG.
Table 1 shows an input / output truth table of the R-S flip-flop 1614.

The operation of the zero correlation removal signal generation circuit 16 having the above configuration will be described with reference to FIG.
First, First, the reset signal Reset (Rst is an inverted signal) is to have 0 (t1), the voltage Vc of the capacitor 50 of the correlation detection circuit 10 (see FIG. 14) is the power supply voltage V _d. Then, after setting the reset signal to 1 (t2), comparison pulse signals OUT + (LOUT +), OUT output from the pulse signal comparison circuit 9 on the left eye side to A and B of the zero correlation removal signal detection circuit 161 for the left eye -(LOUT-) is input, and comparison pulse signals OUT + (ROUT +) and OUT- (ROUT-) output from the right eye side pulse signal comparison circuit 9 to A and B of the zero-correlation removal signal detection circuit 161 for the right eye. ) Is entered.

  The comparison pulse signal on the right eye side will be described. When one of ROUT + and ROUT− is 1 and the other is 0 (t3 to t5, t12 to t13, t21 to t22, t34 to t36, t44 to t45), R / Q is 0 at t3-t10, 1 at t10-t12, 0 at t12-t19, 1 at t19-t21, 0 at t21-t26, 1 at t26-t34, 0 at t34-t40, 0 at t40-t44 1, 0 at t44. As for the outputs of the selection circuits 162 and 163, when the SetOut output is 1 (when the R / Q output is 0 or the Act value is 0), ROUT + and ROUT− are output as they are, but the SetOut output is 0 (R / Q When the output is 1 and the Act value is 1, a predetermined RH / L is output.

  Similarly, the comparison pulse signal on the left eye side will be described. When one of LOUT + and LOUT− is 1 and the other is 0 (t4 to t6, t14 to t15, t21 to t22, t33 to t35, t42 to t43). L / Q is 1 at t4, 0 at t4 to t10, 1 at t10 to t14, 0 at t14 to t19, 1 at t19 to t21, 0 at t21 to t26, 1 at t26 to t33, t33 to 0 at t40, 1 at t40 to t42, and 0 at t42. The outputs of the selection circuits 162 and 163 are LROUT + and LOUT− as they are when the SetOut output is 1 (when the L / Q output is 0 or the Act value is 0), but the SetOut output is 0 (L / Q output) Is 1 and the Act value is 1), a predetermined LH / L is output.

The outputs of RMOUT +, RMOUT−, LMOUT +, LMOUT− are input to the terminals R +, R−, L +, L− of the correlation detection circuit 10 in FIG.
Firstly, first, a reset signal (Reset) is a 0 (t1), the voltage Vc of the capacitor 50 is a power supply voltage V _d. Then, after setting the reset signal to 1 (t2), the comparison pulse signals LMOUT + and LMOUT− output from the zero-correlation removal signal detection circuit 161 on the left eye side are input to L + and L−, and the right eye is input to R + and R−. Comparison pulse signals RMOUT + and RMOUT− output from the zero correlation removal signal generation circuit 161 on the side are input.

  When one of R + and L + is 1 and the other is 0 (t3 to t4, t5 to t6, t12 to t13, t14 to t15), the current switch circuit 51 becomes conductive. Accordingly, at this time, the electric charge of the capacitor 50 is discharged, and the voltage of the capacitor 50 decreases.

  When one of R- and L- is 1 and the other is 0 (t28 to t29, t29 to t31, t33 to t34, t35 to t36, t42 to t43, t44 to t45), the current switch circuit 52 is conductive. It becomes a state. Accordingly, at this time, the electric charge of the capacitor 50 is discharged, and the voltage of the capacitor 50 decreases.

  When R + and L + are both 0 or 1, and both R− and L− are 0 or 1 (t1 to t3, t4 to t5, t21 to t22, t34 to t35, t36 to t40, t43 to t44, At t45-), the current switch circuits 51 and 52 are both cut off. Accordingly, at this time, the voltage Vc of the capacitor 50 is constant.

  After the ramp signal falls, the voltage Vc of the capacitor 50 is determined before the Read signal rises. When the correlation between R + and L + is small, or when the correlation between R− and L− is small, the final voltage Vc of the capacitor 50 becomes low. Conversely, when the correlation between R + and L + is large, or when the correlation between R− and L− is large, the final voltage Vc of the capacitor 50 is kept high. The correlation circuit Vc applied by the zero correlation removal signal generation circuit in place of the H / L signal is forced to have a low value.

  After the voltage Vc of the capacitor 50 is determined, the read signal (read) becomes 1 (t9, t18, t25, t30, t39, t47), and the read switch 56 becomes conductive. As a result, the output circuit 55 outputs a current having a magnitude proportional to the voltage Vc of the capacitor 50.

  After the output is completed, the reset signal is set to 0 again (t10, t19, t26, t31, t40), and the same correlation detection calculation is repeated.

  As described above, a predetermined signal is output when the zero signal detection circuit 161 detects the comparison pulse signal that is the output of + and − of the pulse signal comparison circuit 9, and the comparison pulse signal otherwise. Therefore, the correlation data is a characteristic value only for the object portion. Therefore, it is possible to detect the position only by simple threshold processing, and the calculation cost of post-processing can be greatly reduced. As a result, the cost of the apparatus can be reduced and the processing time can be shortened.

  The parallax correlation data obtained by the parallax sensor according to the present embodiment described above can form a characteristic pattern c where the object a exists, as shown in FIG. By detecting, it is possible to specify the horizontal position and depth of the object (distance to the object).

  INDUSTRIAL APPLICABILITY The present invention can be used in fields such as a three-dimensional motion detection device and a monitoring device as a parallax sensor and a parallax image generation method that can easily extract distance information based on stereo vision.

It is a figure showing the structure of the parallax sensor which concerns on embodiment of this invention. It is a circuit diagram which shows the structure of the zero correlation removal signal generation circuit which concerns on embodiment of this invention. It is a circuit diagram which shows the structure of the zero signal detection circuit which concerns on embodiment of this invention. It is a circuit diagram which shows the structure of the RS flip-flop circuit which concerns on embodiment of this invention. It is a time chart which shows the operation example of the correlation detection circuit which introduce | transduced the zero correlation removal function which concerns on embodiment of this invention. It is a chart which shows the example of correlation data concerning an embodiment of the invention. It is a figure which shows the principle of the distance detection by a binocular parallax. It is a figure which shows the structure of the parallax sensor proposed in patent document 1. FIG. FIG. 9 is a diagram showing a more detailed circuit configuration of the parallax sensor of FIG. 8. It is a figure showing the structure of a voltage and pulse width conversion circuit. It is a time chart which shows the operation example of a voltage and pulse width conversion circuit. It is a figure showing the structure of a pulse signal comparison circuit. It is a time chart showing the operation example of a pulse signal comparison circuit. It is a figure showing the structure of a correlation detection circuit. It is a time chart showing the operation example of a correlation detection circuit. It is a time chart showing the operation example of a parallax sensor. 10 is a diagram illustrating another configuration of a pulse signal comparison circuit of a parallax sensor described in Patent Literature 1. FIG. 10 is a chart showing an example of correlation data by a parallax sensor described in Patent Document 1.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Parallax sensor 2a, 2b Image pick-up element 3 Sequencer 4a, 4b Voltage / pulse width conversion circuit array 5a, 5b Pulse signal comparison circuit array 6 Correlation detection circuit matrix 7 Sequencer 8 Voltage / pulse width conversion circuit 9, 9 'Pulse signal comparison circuit (DIFC)
10 Correlation detection circuit (MATC)
11 Current-voltage converter (IVC)
12 shift register 13 synchronization control circuit 15a, 15b zero correlation removal signal generation circuit array 16 zero correlation removal signal generation circuit 161 zero signal detection circuit 162, 163 selection circuit 1611 NOR circuit 1612 NOT circuit 1613 MOSFET
1614 R-S flip-flop 1615 NAND circuit 21 Channel size adjustable MOS transistor (VS-MOS)
22 Source 22a, 23a, 24a, 25a, 26a Contact hole 23 Drain 24 Main gate 25, 26 Control gate 25b, 26b Gap 31 VS-pMOS
32 VS-nMOS
41, 42, 45, 46 Inverter 43, 44, 47, 48 AND gate 50 Capacitor 51, 52 Current switch circuit 53 Current source 54 Reset switch 55 Output circuit 56 Read switch

Claims (3)

A first image sensor that images a target and outputs a first image as a pixel signal that is an analog voltage signal;
A second imaging element that images the object from an angle different from that of the first imaging element and outputs a second image as a pixel signal that is an analog voltage signal;
A plurality of voltage / pulse width conversion circuits for converting each of the pixel signals output from the first and second image sensors into a pulse width pixel signal having a pulse width having a length proportional to the voltage value of each pixel signal. When,
A synchronous control circuit that performs timing control so that all the voltage / pulse width conversion circuits simultaneously convert each pixel signal into a pulse width pixel signal in parallel;
A plurality of pulse signals that compare two pulse width pixel signals output from the adjacent voltage / pulse width conversion circuits and output a difference between two pulse width pixel signals in a positive direction and a negative direction as comparison pulse signals, respectively. A comparison circuit;
Two comparison pulse signals each composed of a combination of a comparison pulse signal that is an output of the pulse signal comparison circuit corresponding to the first image and a comparison pulse signal that is an output of the pulse signal comparison circuit corresponding to the second image In contrast, multiple correlation detections that convert the total pulse length of the differential pulse obtained by taking the exclusive OR of both into a signal of voltage value or current value proportional to the total pulse length and output this signal as a correlation signal Circuit,
In a parallax sensor comprising:
A zero signal detection circuit for detecting that the pulse width of the comparison pulse signal, which is an output of the pulse signal comparison circuit, is equal to or less than a certain length, and the comparison pulse signal or the predetermined signal according to the output signal of the zero signal detection circuit parallax sensors, characterized in that selects a signal with a zero decorrelation signal generation circuits comprising a selection circuit for outputting the correlation detection circuit.
  2. The parallax sensor according to claim 1, further comprising a function of inputting a comparison pulse signal to the zero signal detection circuit via a MOSFET and adjusting a pulse width detected by a voltage value applied to a gate of the MOSFET. . The first image pickup device picks up an image of the object and outputs a first image as a pixel signal that is an analog voltage signal. At the same time, the second image pickup device picks up the object from an angle different from that of the first image pickup device. A first step of outputting the second image as a pixel signal that is an analog voltage signal;
Each pixel signal output from the first and second imaging elements is pulsed with a pulse width having a length proportional to the voltage value of the pixel signal simultaneously and in parallel by a plurality of voltage / pulse width conversion circuits. A second step of converting to a pixel signal;
A third step of comparing two pulse width pixel signals output from the adjacent voltage / pulse width conversion circuits, and outputting a difference between the two pulse width pixel signals in the positive direction and a negative direction as comparison pulse signals, respectively. When,
A fourth step of selecting and outputting the comparison pulse signal or a predetermined signal when it is detected that the pulse width of the comparison pulse signal is equal to or less than a certain length;
A comparison pulse signal corresponding to the first image is selected by a plurality of correlation detection circuits. However, when a predetermined signal is selected in the fourth step, the predetermined signal corresponds to the second image. However, when a predetermined signal is selected in the fourth step, the exclusive OR of the two pulse width pixel signals each consisting of a combination of the predetermined signals is selected. A fifth step of converting a total pulse length of the differential pulse obtained by taking a signal of a voltage value or a current value proportional to the total pulse length and outputting this signal as a correlation signal;
A method for generating a parallax image.