JP5950193B2 - Disparity value calculation device, disparity value calculation system including the same, moving surface area recognition system, disparity value calculation method, and disparity value calculation program - Google Patents

Description

  The present invention relates to a parallax value calculation device that calculates a parallax value between a pair of captured image data obtained by imaging an imaging region around a moving body by a pair of imaging means mounted on the moving body, and a parallax value provided with the parallax value The present invention relates to a calculation system, a moving surface area recognition system, a parallax value calculation method, and a parallax value calculation program.

  This type of parallax value calculation device is a mobile control device that controls the movement of a mobile object such as a vehicle, ship, aircraft, or industrial robot, or an information providing device that provides useful information to a driver of the mobile object. Widely used in object detection processing. If a specific example is given, what is used for driver support systems, such as ACC (Adaptive Cruise Control) for reducing the driving load of the driver (driver) of vehicles, for example is known. In such a driver assistance system, an automatic brake function and an alarm function for avoiding the collision of the own vehicle with an obstacle or reducing an impact at the time of the collision, and maintaining a distance from the preceding vehicle are maintained. Various functions are realized, such as a function for adjusting the speed of the own vehicle to support the vehicle and a function for supporting the prevention of deviation from the traveling lane in which the host vehicle is traveling. For this purpose, an image captured around the host vehicle is analyzed, and various objects existing around the host vehicle (for example, other vehicles, pedestrians, road surface components such as lanes and manhole covers, electric poles, guardrails, etc. It is important to accurately detect the distance to the roadside structure.

  In Patent Document 1, an infrared image of a distance measurement target (parallax value calculation target) is captured by two cameras mounted on a vehicle at the same height in the horizontal direction, and the parallax of two infrared images captured by each camera. An image distance measuring device that calculates a distance to a distance measurement object using a value is disclosed. In this image distance measuring device, one image of two cameras is used as a reference image, the reference image is divided into pattern areas, and the pattern area of the same size is sequentially shifted on the other image while the two images are in between. Correlation calculation of the pattern area is performed, and a shift amount that maximizes the degree of correlation is obtained. Then, the distance to the distance measurement object displayed in the pattern area is calculated from the parallax values of the two images, which are the shift amounts, by the principle of triangulation.

FIGS. 1A and 1B are explanatory diagrams illustrating examples of captured images respectively captured by a pair of cameras.
In the example shown in FIGS. 1A and 1B, first, a pair of cameras having the same epipolar lines are used, and the same imaging region is imaged from different directions, and of the two captured images obtained thereby. One (image shown in FIG. 1A) is a reference image, and the other (image shown in FIG. 1B) is a comparative image. Thereafter, it is searched which image region in the reference image corresponds to an image region in the reference image. For example, when searching for an image area in the comparative image corresponding to the image area Wa in the reference image, the horizontal position is changed while the vertical position is fixed at the same position as the vertical position of the image area Wa. The comparison image is searched in the order of Wb1 → Wb2 → Wb3. Then, while performing this search, the correlation calculation is performed between the reference image and the comparison image, the image area having the highest degree of correlation is detected, and the corresponding point corresponding to the same point in the imaging area is compared with the reference image. A matching process that specifies both images is performed.

FIG. 2 is an explanatory diagram showing a general distance measuring principle.
When the image processing region Wb3 in the comparison image corresponding to the image region Wa in the reference image is specified by the matching process, the corresponding points on the image regions Wa and Wb3 corresponding to the same point in the imaging region are compared with the reference image. The amount of deviation from the image is obtained as a parallax value, and the distance to the same point that is the object of distance measurement is calculated using the principle of triangulation.

  When calculating a parallax value between a pair of captured image data captured by a pair of cameras (imaging means) mounted on a moving body, the pair of captured image data is desirably captured at the same imaging timing. It is. This is because the parallax value is calculated on the assumption that the image shift between the pair of captured image data is caused only by the difference between the attachment positions of the pair of cameras.

FIGS. 3A and 3B are image examples schematically showing examples of captured images obtained by capturing the front in the traveling direction of the host vehicle with a pair of cameras whose imaging timings are shifted.
In this example, the captured image illustrated in FIG. 3A is captured first, and the captured image illustrated in FIG. 3B is captured later. When the imaging timing of one camera is delayed in this way, as shown in the figure, the host vehicle on which the pair of cameras are mounted moves during the delay time, and the pair of cameras also moves accordingly. To do. Therefore, the image shift between the pair of captured image data includes a shift due to the movement of the camera during the delay time. For example, when the parallax value calculation target moves such as another vehicle or a pedestrian, the parallax value calculation target moves during the delay time, and the parallax value calculation between a pair of captured image data is performed. The shift of the target image includes a shift caused by the movement of the parallax value calculation target.

  As described above, when the imaging timing of the pair of cameras is misaligned, the image misalignment between the pair of captured image data includes a misalignment based on a factor that is different from the difference in the attachment position of the pair of cameras. In that case, when the parallax value is calculated from the shift of the image, there arises a problem that a different parallax value is calculated from the original parallax value for the image so as to exceed an allowable range.

  In order to solve the above problem, the present inventor detects the amount of deviation of the imaging timing between a pair of cameras, and restricts the use of the parallax value when the detected amount of deviation exceeds a predetermined allowable deviation range. A working parallax value calculation device was developed. According to this parallax value calculation device, even when an imaging timing deviation that exceeds an allowable range occurs, the use of an incorrect parallax value is restricted, so that the occurrence of a malfunction due to an incorrect parallax value can be suppressed. Then, if the shift of the imaging timing between the pair of cameras is returned within the allowable range by adjusting the synchronization signal of the cameras, an appropriate parallax value can be used again.

  However, the optimum allowable range of the shift amount of the imaging timing between the pair of cameras varies depending on the moving speed of the moving body on which the pair of cameras is mounted. That is, even if the amount of deviation of the imaging timing is constant, if the speed of the moving body changes, the distance that the moving body moves during the time corresponding to the amount of deviation changes. Further, when the parallax calculation target is moving, if the speed of the moving body changes, the distance that the parallax calculation target moves during the time corresponding to the amount of deviation changes. Therefore, when the speed of the moving body changes, even if the amount of shift in the imaging timing is constant, the effect of the movement of the moving body on the image shift between the pair of captured image data changes, and this is calculated from the shift amount of this image. The error of the parallax value is changed.

  The allowable range of the amount of deviation of the imaging timing is set on the basis that the calculated parallax value error does not exceed the allowable range. Therefore, the optimum allowable range of the amount of deviation of the imaging timing changes according to the speed of the moving object even if the amount of deviation of the imaging timing is constant. Therefore, in the processing operation that does not output the parallax value when the deviation amount of the imaging timing exceeds a predetermined allowable deviation range, an erroneous parallax occurs when the speed of the moving object is faster than expected. There arises a problem that the use of values cannot be stably restricted. On the other hand, if the constant deviation allowable range is set to a sufficiently narrow range in advance, the use of an erroneous parallax value can be stably limited, but even with a parallax value that includes only a sufficiently allowable error, Use of the value is limited, which causes a problem in practical use.

  The present invention has been made in view of the above background, and the object of the present invention is to set an allowable deviation range in an excessively narrow range even when a deviation occurs in imaging timing between a pair of cameras. Provided are a parallax value calculation device that can stably limit the use of an incorrect parallax value, a parallax value calculation system including the same, a moving surface area recognition system, a parallax value calculation method, and a program for calculating a parallax value It is to be.

To achieve the above object, the present invention is an imaging area around the mobile body captured by the pair of imaging means to be mounted on the movable body having a moving speed detecting means for detecting a moving speed of the moving body In the parallax value calculating device including the parallax value calculating means for calculating the parallax value between the pair of captured image data obtained in this manner, information indicating each imaging timing of the pair of imaging means input from the pair of imaging means A timing shift amount detecting means for detecting a shift amount of an imaging timing between the pair of image pickup means, and the disparity value when the shift amount detected by the timing shift amount detecting means exceeds a predetermined shift allowable range. Use restriction processing execution means for executing processing for restricting the use of the parallax value calculated by the calculation means, and the use restriction processing execution means is detected by the moving speed detection means. Based on the degree detection result, as the moving speed of the moving body is spread slowly that the predetermined deviation tolerance, and changes the predetermined deviation tolerance.

  The influence of the movement of the moving body on the image shift between the pair of captured image data becomes larger as the speed of the moving body increases. Therefore, in the present invention, the allowable shift range of the imaging timing between the pair of imaging units is changed so that the predetermined allowable shift range is widened as the moving speed of the moving body is slow. Thereby, even if the error of the parallax value changes according to the speed of the moving object, the allowable range of deviation of the imaging timing is changed so that the error does not exceed the allowable range. Therefore, even if the allowable range of imaging timing deviation is not set to an excessively narrow range in advance, it is possible to stably limit the use of an incorrect parallax value.

  According to the present invention, even when a deviation occurs in the imaging timing between a pair of cameras, it is possible to stably limit the use of an incorrect parallax value without setting the deviation allowable range to an excessively narrow range. An excellent effect is obtained.

(A) And (b) is explanatory drawing which shows the reference image and comparative image which were each imaged with the two cameras which comprise a stereo camera. It is explanatory drawing which shows the general ranging principle. (A) And (b) is an example of an image which showed typically an example of a picked-up image which picturized the direction of advancing of a self-vehicle with a pair of cameras from which imaging timing shifted, respectively. It is a mimetic diagram showing a schematic structure of an in-vehicle device control system in an embodiment. It is a schematic diagram which shows schematic structure of the imaging unit and image analysis unit in the same vehicle equipment control system. (A) is explanatory drawing which shows an example of the parallax value distribution of a parallax image. (B) is explanatory drawing which shows the row parallax distribution map (V map) which shows the parallax value frequency distribution for every row | line | column of the parallax image of the same (a). (A) is an image example schematically representing an example of a captured image (luminance image). (B) is a graph obtained by linearly approximating the row parallax distribution map (V map) calculated by the parallax histogram calculation unit. It is a flowchart which shows the flow of the road surface area recognition process in embodiment. It is a timing chart for demonstrating the output data system of each imaging part. It is a timing chart for demonstrating operation | movement of the phase difference detection part in embodiment.

Hereinafter, an embodiment in which a parallax value calculation device according to the present invention is used in an in-vehicle device control system will be described.
In addition, the parallax value calculation apparatus according to the present invention is not limited to the in-vehicle device control system, and can be applied to, for example, other systems equipped with an object detection apparatus that detects an object based on a captured image.

FIG. 4 is a schematic diagram showing a schematic configuration of the in-vehicle device control system in the present embodiment.
This in-vehicle device control system recognizes a recognition target object using captured image data of a forward area (imaging area) in the traveling direction of the host vehicle captured by an imaging unit mounted on the host vehicle 100 that is a moving body such as an automobile. It controls various in-vehicle devices according to the above.

  The in-vehicle device control system of the present embodiment is provided with an image pickup unit 101 that picks up an image of an area in the traveling direction of the traveling vehicle 100 as an image pickup area. For example, the imaging unit 101 is installed in the vicinity of a room mirror (not shown) of the windshield 105 of the host vehicle 100. Various data such as captured image data obtained by imaging by the imaging unit 101 is input to the image analysis unit 102. The image analysis unit 102 analyzes the data transmitted from the imaging unit 101 and detects, for example, a road surface region (moving surface region) that displays a road surface (moving surface) on which the host vehicle 100 is traveling.

  The calculation result of the image analysis unit 102 is sent to the vehicle travel control unit 108. Based on the recognition result of the road surface area (movable area) detected by the image analysis unit 102, the vehicle travel control unit 108 notifies the driver of the own vehicle 100 when the own vehicle 100 is likely to deviate from the travelable area. A warning is notified, and driving support control such as controlling the steering wheel and brake of the host vehicle is performed.

FIG. 5 is a schematic diagram showing a schematic configuration of a road surface area recognition system (moving surface recognition system) including a parallax value calculation system used in the in-vehicle device control system of the present embodiment.
The imaging unit 101 is a stereo camera provided with a pair of two imaging units 110A and 110B as imaging means, and the configuration of the two imaging units 110A and 110B is the same. Each of the imaging units 110A and 110B mainly includes an imaging lens, an optical filter as necessary, a sensor board including an image sensor in which an imaging element is two-dimensionally arranged, and an analog electrical signal ( A signal processing unit that generates and outputs captured image data obtained by converting a received light amount received by each light receiving element on the image sensor into a digital electrical signal.

  In addition, the imaging unit 101 includes a processing hardware unit including an FPGA (Field-Programmable Gate Array) or the like. The processing hardware unit obtains parallax image data from a pair of captured image data output from the imaging units 110A and 110B, and the parallax of the corresponding image portion between the captured images captured by the imaging units 110A and 110B. A parallax calculation unit 121 is provided as parallax value calculation output means for calculating and outputting values. The parallax value here refers to a comparison image for an image portion on the reference image corresponding to the same point in the imaging region, with one of the captured images captured by each of the imaging units 110A and 110B as a reference image and the other as a comparison image. The positional deviation amount of the upper image part is calculated as the parallax value of the image part. By using the principle of triangulation, the distance to the same point in the imaging area corresponding to the image portion can be calculated from the parallax value.

The method for calculating the distance from the parallax value is as follows.
The parallax value calculated by the parallax calculation unit 121 (the amount of deviation between the image portion on the reference image and the corresponding image portion on the comparison image) is d, and the interval (baseline length) between the two imaging units 110A and 110B is B. When the focal length is f, the distance Z to the object displayed in the image portion can be calculated from the following equation (1). The distance Z calculated in this way may be displayed as a numerical value on an image display device in the vehicle interior.
Z / B = f / d (1)

  On the other hand, the image analysis unit 102 includes a memory that stores parallax image data output from the imaging unit 101, and an MPU (Micro Processing Unit) that includes software for performing recognition processing of an identification target, parallax calculation control, and the like. ing. The MPU executes various recognition processes using the parallax image data stored in the memory.

Next, the road surface recognition process performed by the road surface area recognition system in the present embodiment will be described.
The parallax calculation unit 121 of the present embodiment uses captured image data of one imaging unit 110A of the two imaging units 110A and 110B as reference image data, and uses captured image data of the other imaging unit 110B as comparison image data. Then, the parallax values of both are calculated to generate and output parallax image data. This parallax image data indicates a parallax image in which pixel values corresponding to the parallax values calculated for each image portion on the reference image data are represented as pixel values of the respective image portions.

  Specifically, the parallax calculation unit 121 defines a block including a plurality of pixels (for example, 16 pixels × 1 pixel) centered on one target pixel for a certain row of reference image data. On the other hand, in the same row in the comparison image data, a block having the same size as the block of the defined reference image data is shifted by one pixel in the horizontal line direction (X direction) to show the feature of the pixel value of the block defined in the reference image data. Correlation values indicating the correlation between the feature amount and the feature amount indicating the feature of the pixel value of each block in the comparison image data are calculated. Then, based on the calculated correlation value, a matching process is performed for selecting a block of comparison image data that is most correlated with a block of reference image data among the blocks of the comparison image data. Thereafter, a positional deviation amount between the target pixel of the block of the reference image data and the corresponding pixel of the block of the comparison image data selected by the matching process is calculated as a parallax value. The parallax image data can be obtained by performing such a process of calculating the parallax value for the entire area of the reference image data or a specific area. The parallax image data obtained in this way is sent to the parallax histogram calculation unit 141.

  As the feature amount of the block used for the matching process, for example, the value (luminance value) of each pixel in the block can be used, and as the correlation value, for example, the value (luminance) of each pixel in the block of the reference image data Value) and the sum of absolute values of the differences between the values (luminance values) of the pixels in the block of comparison image data corresponding to these pixels, respectively. In this case, it can be said that the block having the smallest sum is most correlated.

  The parallax histogram calculation unit 141 that acquired the parallax image data calculates the parallax value frequency distribution for each row of the parallax image data. Explaining with a specific example, when parallax image data having a parallax value distribution as shown in FIG. 6A is input, the parallax histogram calculation unit 141 performs line by row as shown in FIG. The disparity value frequency distribution is calculated and output. From the parallax value frequency distribution information of each row obtained in this way, for example, the vertical position on the parallax image is taken in the vertical direction and the parallax value is taken in the horizontal direction on the parallax image data. A row parallax distribution map (V map) in which each pixel is distributed can be obtained.

FIG. 7A is an image example schematically illustrating an example of a captured image (luminance image) captured by the imaging unit 110A. FIG. 7B illustrates each row calculated by the parallax histogram calculation unit 141. 5 is a graph obtained by linearly approximating the pixel distribution on the row parallax distribution map (V map) from the parallax value frequency distribution of the first parallax.
The image example shown in FIG. 7A is an image of the situation where the host vehicle is traveling in the left lane on a straight road with two lanes on one side, and the reference CL in the figure indicates the central lane. This is a median strip image portion to be projected, and in the figure, a symbol WL is a white line image portion (lane border image portion) that projects a white line that is a lane boundary line, and a symbol EL in the drawing is a step such as a curb on the road edge. This is a road edge step image portion to be projected. Hereinafter, the road edge step image portion EL and the central separation band step image portion CL are collectively referred to as a step image portion. Moreover, area | region RS enclosed with the broken line in the figure is a road surface area | region where the vehicle driving | running | working divided by the median strip and a roadside level | step difference is possible.

  In the present embodiment, the road surface area recognition unit 142 recognizes the road surface area RS from the parallax value frequency distribution information of each row output from the parallax histogram calculation unit 141. Specifically, the road surface area recognition unit 142 first acquires the disparity value frequency distribution information of each row from the disparity histogram calculation unit 141, and determines the pixel distribution on the row disparity distribution map specified from the information by using the least square method or the like. A straight line approximation process such as a Hough transform process is performed. The approximate straight line shown in FIG. 7B obtained as a result is a straight line having an inclination such that the parallax value decreases toward the upper side of the image at the lower part of the row parallax distribution map corresponding to the lower part of the parallax image. That is, pixels distributed on or near the approximate line (pixels on the parallax image) are present at almost the same distance in each row on the parallax image, have the highest occupancy, and the distance increases toward the top of the image. It can be said that it is a pixel that projects objects that are continuously distant.

  Here, since the imaging unit 110A captures the front area of the host vehicle, the content of the parallax image has the highest occupation ratio of the road surface area RS in the lower part of the image as shown in FIG. The parallax value of the road surface region RS becomes smaller toward the road. Further, in the same row (horizontal line), the pixels constituting the road surface region RS have substantially the same parallax value. Therefore, the pixels distributed on or near the approximate straight line on the row parallax distribution map (V map) specified from the parallax value frequency distribution information of each row output from the parallax histogram calculation unit 141 are the road surface region RS. It matches the characteristics of the pixels that make up. Therefore, it can be estimated that the pixels distributed on or near the approximate straight line shown in FIG. 7B are pixels constituting the road surface region RS with high accuracy.

  As described above, the road surface area recognition unit 142 of the present embodiment performs linear approximation on the row parallax distribution map (V map) calculated based on the parallax value frequency distribution information of each row obtained from the parallax histogram calculation unit 141, Pixels distributed on or in the vicinity of the approximate straight line are specified as pixels that project the road surface, and an image region occupied by the specified pixels is recognized as a road surface region RS. Although a white line is also present on the road surface as shown in FIG. 7A, the road surface area recognition unit 142 recognizes the road surface area RS including the white line image part WL.

  The recognition result of the road surface area recognition unit 142 is sent to the vehicle travel control unit 108. In the vehicle travel control unit 108, for example, when the host vehicle 100 is likely to come off from the travelable region based on the road surface region recognition result, a warning is given to the driver of the host vehicle 100, or the handle of the host vehicle is Or driving support control such as controlling a brake. Further, in the vehicle travel control unit 108, for example, when the vehicle travel control unit 108 performs control to display an image captured in front of the host vehicle captured by the imaging unit 101 on, for example, an image display device in the host vehicle interior, Based on the recognition result of the road surface area recognition unit 142, the control unit 108 may perform display processing that makes it easy to visually recognize the road surface area RS, such as highlighting the corresponding road surface area RS on the display image.

  Here, when the imaging timings of the pair of imaging units 110A and 110B constituting the stereo camera are shifted, the other imaging unit captures images with a delay from the imaging timing of one imaging unit. Since the host vehicle 100 moves during the delay time, as described above, the parallax value calculated by the parallax calculation unit 121 includes an error due to the host vehicle moving during the delay time. . If this parallax value error exceeds the allowable range, the parallax value frequency distribution information obtained from the parallax value calculated by the parallax calculation unit 121 becomes inaccurate, and the road surface area is calculated from the inaccurate parallax value frequency distribution information. The road surface area RS recognized by the recognition unit 142 has low recognition accuracy. In this case, the reliability of the road surface area recognition result of the road surface area recognition unit 142 becomes low, and there is a possibility that subsequent processing cannot be performed properly.

  Therefore, in this embodiment, the phase difference detection unit 122 detects the phase difference between the synchronization signals of the pair of imaging units 110A and 110B in order to grasp the amount of deviation of the imaging timing between the pair of imaging units 110A and 110B. . Whether or not the phase difference detected by the phase difference detection unit 122 causes an error that exceeds the allowable range with respect to the parallax value calculated by the parallax calculation unit 121 in the synchronization deviation determination unit 123. Judging. As a result of this determination, when it is determined that an error exceeding the allowable range is caused with respect to the parallax value, the road surface area recognition unit 142 does not use the parallax value frequency distribution information from the parallax histogram calculation unit 141. The road surface area RS is recognized from the white line recognition result of the white line recognition processing unit 149.

  Specifically, the white line recognition processing unit 149 acquires captured image data from the imaging unit 110A that captures the reference image data, and first extracts a portion where the pixel value (luminance) changes by a specified value or more as an edge portion. As the edge extraction method, known methods can be widely used. In many roads, a white line is formed on a road surface of a color close to black, and the luminance of the white line image portion WL is sufficiently higher than that of other portions on the road surface on the luminance image. Therefore, there is a high possibility that an edge portion having a pixel value difference equal to or greater than a predetermined value on the luminance image is a white line edge portion. In addition, the white line image portion WL that projects the white line on the road surface is projected in a line shape on the captured image. Therefore, the edge portion of the white line can be recognized with high accuracy by specifying the edge portion arranged in the line shape. . Therefore, the white line recognition processing unit 149 according to the present embodiment performs a straight line approximation process such as a least square method or a Hough transform process on the edge portion, and uses the obtained approximate straight line as a white line edge portion (a white line image that displays a white line on the road surface). Part WL).

  The road surface area recognition unit 142 performs the following process when recognizing the road surface area RS based on the white line recognition result output from the white line recognition processing unit 149. First, for the captured image data, an edge portion is searched for outward in the left-right direction of the image from the white line image portion specified based on the white line recognition result. In many cases, a road side member such as a step or a wall exists at a side end portion (outside the white line) of the road surface, and a luminance difference is generated at a boundary with the road surface due to a shadow generated by the road side member. Therefore, it is highly possible that the edge portion that exists outside the white line image portion in the left-right direction of the image is such a shaded portion, and this can be recognized as a boundary of the road surface region.

Next, the road surface area recognition process of the road surface area recognition unit 142 using the amount of deviation of the image pickup timing of the pair of image pickup units 110A and 110B will be described.
FIG. 8 is a flowchart showing a flow of road surface area recognition processing in the present embodiment.
In the present embodiment, the vertical synchronization signal and the horizontal synchronization signal are input from the pair of imaging units 110A and 110B to the phase difference detection unit 122, and the phase difference between the vertical synchronization signals is detected (S1). . The unit of measurement of the phase difference is the number of horizontal synchronization signals. Therefore, the shift in the imaging timing of the present embodiment can be grasped in such a manner that, for example, the shift is equivalent to seven horizontal synchronization signals.

  The phase difference detection unit 122 according to the present embodiment includes an up / down counter, and represents a phase difference between synchronization signals output from the pair of imaging units 110A and 110B as a count value. More specifically, the time difference from when the vertical synchronization signal of one imaging unit becomes H level at a certain time until the vertical synchronization signal of the other imaging unit becomes H level is measured.

Prior to the description of the operation of the phase difference detection unit 122, the output data system of each of the imaging units 110A and 110B will be described.
FIG. 9 is a timing chart for explaining the output data system of each of the imaging units 110A and 110B.
Image data is continuously output from each of the imaging units 110A and 110B. A vertical synchronization signal indicating where and where the image data corresponds to one screen, and one to horizontal lines from where to where in the image data ( A horizontal synchronizing signal indicating whether it corresponds to one line) is also output. The image data includes not only effective image data obtained by photographing the imaging region, but also data that is necessary for the operation of the imaging unit but has no meaning as an image. The horizontal synchronization signal indicates an H level during a period in which a valid image is output in one horizontal line. The vertical synchronization signal indicates the H level during a period in which the horizontal synchronization signal is continuously output.

FIG. 10 is a timing chart for explaining the operation of the phase difference detection unit 122 in the present embodiment.
The up / down counter constituting the phase difference detection unit 122 is one of the first vertical synchronization signal output from one imaging unit 110A and the second vertical synchronization signal output from the other imaging unit 110B. When a transition from L level to H level is detected, the count valid period starts. When the first vertical synchronization signal transits to H level and the count valid period starts, a count-up operation is performed. On the other hand, when the second vertical synchronization signal transitions to the H level and enters the count valid period, the countdown operation is performed. The illustrated example is a case where the first vertical synchronization signal transitions to the H level and enters the count valid period, so the up / down counter performs a count-up operation.

  Thereafter, when the up / down counter detects that the other vertical synchronization signal has transitioned from the L level to the H level, it ends the count valid period and maintains the count value at that time. Thereafter, when it is detected that one of the first vertical synchronization signal and the second vertical synchronization signal has transitioned from the H level to the L level, the count value is returned to zero.

  Either the count-up operation or the count-down operation in the count valid period is performed once, either the first horizontal synchronization signal output from the imaging unit 110A or the second horizontal synchronization signal output from the imaging unit 110B. Every time an input is made, one count is performed. Therefore, the count value of the up / down counter indicates how many times the horizontal synchronizing signal corresponds to the phase difference between the vertical synchronizing signals. In other words, it can be said that it indicates how many horizontal lines the deviation of the start time of one screen corresponds to.

  The shift amount of the imaging timing detected as described above (the count value of the phase difference detection unit 122 = the phase difference of the vertical synchronization signal) is sent to the synchronization shift determination unit 123. The synchronization shift determination unit 123 determines whether the shift amount of the imaging timing is large enough to hinder the matching process described above, and outputs the determination result to the road surface area recognition unit 142.

  Prior to the determination, the synchronization deviation determination unit 123 first acquires vehicle speed information detected by a vehicle speed detection unit (not shown) mounted on the host vehicle 100 (S2), and based on this vehicle speed information, determines the imaging timing. A threshold for comparison with the amount of deviation is determined (S3). Specifically, the threshold value is determined so that the threshold value increases as the vehicle speed of the host vehicle 100 decreases. Therefore, the lower the vehicle speed of the host vehicle 100 is, the larger the threshold is, and the permissible deviation range in which the road surface area recognition process using the parallax value frequency distribution information from the parallax histogram calculation unit 141 can be executed is widened. On the contrary, since the threshold value decreases as the vehicle speed of the host vehicle 100 increases, the allowable deviation range in which the road surface area recognition process using the parallax value frequency distribution information from the parallax histogram calculation unit 141 can be executed becomes narrower. In the example illustrated in FIG. 10, the absolute value of the threshold is determined to be 5.

  The synchronization deviation determination unit 123 compares the threshold value determined according to the vehicle speed information in this way with the phase difference (the amount of imaging timing deviation) of the vertical synchronization signal indicated by the count value of the phase difference detection unit 122 ( S4) When the phase difference is equal to or smaller than the threshold value (No in S4), the determination result to that effect is output to the road surface area recognition unit 142. Thereby, the road surface area recognition unit 142 performs the recognition process of the road surface area RS using the parallax value frequency distribution information from the parallax histogram calculation unit 141 (S5).

  On the other hand, when the phase difference exceeds the threshold value (Yes in S4), a determination result to that effect is output to the road surface area recognition unit 142. Thereby, the road surface area recognition unit 142 recognizes the road surface area RS from the white line recognition result of the white line recognition processing unit 149 without using the parallax value frequency distribution information from the parallax histogram calculation unit 141 (S6). As described above, when the phase difference exceeds the threshold value, that is, when the deviation amount of the imaging timing exceeds a predetermined deviation allowable range, the pair of imaging is performed so that the deviation amount of the imaging timing returns to the deviation allowable range. It is preferable to execute a processing operation for correcting the imaging timing of the units 110A and 110B.

  In this embodiment, when the phase difference exceeds the threshold value, that is, when the deviation amount of the imaging timing exceeds a predetermined deviation tolerance range, the luminance information is used instead of the road surface recognition process based on the parallax value. Although the road surface recognition process based on this is performed, for example, a subsequent process itself using the parallax value (such as a road surface recognition process or a process of calculating a distance from the parallax value) may be stopped.

What has been described above is merely an example, and the present invention has a specific effect for each of the following modes.
(Aspect A)
Imaging around the moving body by an imaging unit such as a pair of imaging units 110A and 110B mounted on the moving body provided with a moving speed detecting unit such as a vehicle speed detecting unit for detecting the moving speed of the moving body such as the host vehicle 100 In a parallax value calculation device including a parallax value calculation unit such as a parallax calculation unit 121 that calculates a parallax value between a pair of captured image data obtained by imaging a region, the pair of images input from the pair of imaging units Timing shift amount detection means such as a phase difference detection unit 122 for detecting a shift amount of the imaging timing between the pair of imaging means using information such as a synchronization signal indicating each imaging timing of the imaging means, and the timing shift amount detection When the amount of deviation detected by the means exceeds a predetermined deviation allowable range, a synchronization deviation determination unit that executes processing for limiting the use of the parallax value calculated by the parallax value calculating means 23, etc., and the use restriction processing execution means is based on the speed detection result detected by the movement speed detection means, and the predetermined deviation allowable range as the moving speed of the moving body is slow. The predetermined deviation allowable range is changed so as to spread.
According to this, even if the error of the parallax value changes according to the speed of the moving body, the allowable deviation range of the imaging timing is changed so that the error does not exceed the allowable range. Even if an excessively narrow range is not set in advance, the use of an erroneous parallax value exceeding the allowable range can be stably limited.

(Aspect B)
In the aspect A, a synchronization signal of each captured image data output from the pair of imaging units is used as information indicating each imaging timing of the pair of imaging units. The phase difference of the synchronization signal between the pair of imaging means is detected as the amount of deviation of the imaging timing between the means.
According to this, it is possible to detect a shift amount between imaging timings using a synchronization signal generally used in the imaging unit.

(Aspect C)
In the aspect D, when the synchronization signal for the one imaging unit of the pair of imaging units is first input, the timing shift amount detection unit calculates the phase difference of the synchronization signal between the pair of imaging units. The count value shown is counted up until the synchronization signal for the other imaging means of the pair of imaging means is input, and when the synchronization signal for the other imaging means is input first, the count value Is counted down until a synchronization signal for the one image pickup means is input, and the counted value is output.
According to this, if the count value is larger than the initial value, it can be understood that the imaging timing of the other imaging unit is delayed with respect to one imaging unit, and the count value is smaller than the initial value. If so, it can be understood that the imaging timing of one imaging means is delayed with respect to the other imaging means. Therefore, it is possible to grasp which imaging means of the imaging timing is delayed with a simple configuration.

(Aspect D)
A pair of imaging means mounted on the moving body provided with a moving speed detecting means for detecting the moving speed of the moving body, and a pair of imaging obtained by imaging an imaging area around the moving body by the pair of imaging means In a parallax value calculation system including a parallax value calculation device that calculates a parallax value between image data, the parallax value calculation device according to any one of the modes A to C is used as the parallax value calculation device. It is characterized by.
According to this, a parallax value calculation system that can stably limit the use of an erroneous parallax value exceeding the allowable range without setting the allowable range of deviation of the imaging timing in an excessively narrow range in advance. Can be provided.

(Aspect E)
In a moving surface area recognition system for recognizing a moving surface area on which a moving surface on which the moving body moves is displayed from an imaged image obtained by imaging the periphery of the moving body with an imaging means mounted on the moving body moving on the moving surface. , A process of recognizing the moving surface area based on a parallax value calculation system according to the aspect D, a parallax value calculated by the parallax value calculation system, and a luminance image obtained by at least one of the pair of imaging units. And a recognition processing means for performing.
According to this, even if the allowable range of imaging timing deviation is not set to an excessively narrow range in advance, it is possible to stably limit the use of an erroneous parallax value exceeding the allowable range, Recognition accuracy can be improved.

(Aspect F)
A parallax value between a pair of imaged image data obtained by imaging an imaging region around the moving body by a pair of imaging means mounted on the moving body provided with a moving speed detecting means for detecting the moving speed of the moving body. In the parallax value calculation method including a parallax value calculation step for calculating, a deviation amount of the imaging timing between the pair of imaging units using information indicating each imaging timing of the pair of imaging units input from the pair of imaging units When the amount of deviation detected in the timing deviation amount detection step and the timing deviation amount detection step exceeds a predetermined tolerance, the use of the parallax value calculated in the parallax value calculation step is limited. and a use restriction process executing step of executing the treatment for, the use restriction process executing step, based on the detected speed detection result by the moving speed detecting means, the moving object As the moving speed is slow as the predetermined deviation tolerance spread, and changes the predetermined deviation tolerance.
According to this, it is possible to stably limit the use of an erroneous parallax value exceeding the permissible range without setting the permissible range of the imaging timing in an excessively narrow range in advance.

(Aspect G)
A parallax value between a pair of imaged image data obtained by imaging an imaging region around the moving body by a pair of imaging means mounted on the moving body provided with a moving speed detecting means for detecting the moving speed of the moving body. In a parallax value calculation program for causing a computer to execute a parallax value calculation step to be calculated, the pair of imaging units using information indicating each imaging timing of the pair of imaging units input from the pair of imaging units When the amount of deviation detected in the timing deviation amount detection step exceeds a predetermined allowable range, the calculation is performed in the parallax value calculation step. A use restriction process executing step for executing a process for restricting the use of a parallax value, and executing the use restriction process. Degree, based on the speed detection result detected by the moving speed detection means, as the moving speed of the moving body is spread slowly that the predetermined deviation tolerance, characterized in that to change the predetermined deviation tolerance And
According to this, it is possible to stably limit the use of an erroneous parallax value exceeding the permissible range without setting the permissible range of the imaging timing in an excessively narrow range in advance.
This program can be distributed or obtained in a state of being recorded on a recording medium such as a CD-ROM. It is also possible to distribute and obtain signals by placing this program and distributing or receiving signals transmitted by a predetermined transmission device via transmission media such as public telephone lines, dedicated lines, and other communication networks. Is possible. At the time of distribution, it is sufficient that at least a part of the computer program is transmitted in the transmission medium. That is, it is not necessary for all data constituting the computer program to exist on the transmission medium at one time. The signal carrying the program is a computer data signal embodied on a predetermined carrier wave including the computer program. Further, the transmission method for transmitting a computer program from a predetermined transmission device includes a case where data constituting the program is transmitted continuously and a case where it is transmitted intermittently.

DESCRIPTION OF SYMBOLS 100 Own vehicle 101 Imaging unit 102 Image analysis unit 105 Windshield 108 Vehicle travel control unit 110A, 110B Imaging part 121 Parallax calculating part 122 Phase difference detection part 123 Synchronization shift | offset | difference judgment part 141 Parallax histogram calculation part 142 Road surface area recognition part 149 White line recognition Processing part

JP 2003-28635 A

Claims (7)

Disparity value between mobile pair of a pair of captured image data obtained by imaging the imaging area around the mobile body by the imaging means to be mounted on the mobile body equipped with a moving speed detecting means for detecting a moving speed of the In the parallax value computing device provided with the parallax value computing means for computing
A timing shift amount detecting means for detecting a shift amount of the imaging timing between the pair of imaging means using information indicating each imaging timing of the pair of imaging means input from the pair of imaging means;
Use restriction processing execution means for executing processing for restricting the use of the parallax value calculated by the parallax value calculating means when the amount of deviation detected by the timing deviation amount detecting means exceeds a predetermined deviation allowable range; Have
The use restriction processing execution unit changes the predetermined deviation allowable range based on the speed detection result detected by the moving speed detection unit so that the predetermined deviation allowable range is expanded as the moving speed of the moving body is slower. A parallax value calculation apparatus characterized by: In the parallax value calculation apparatus of Claim 1,
As information indicating each imaging timing of the pair of imaging means, using a synchronization signal of each captured image data output from the pair of imaging means,
The parallax value calculation device, wherein the timing shift amount detection unit detects a phase difference of a synchronization signal between the pair of imaging units as an amount of shift of imaging timing between the pair of imaging units. In the parallax value calculating apparatus of Claim 2,
When the synchronization signal for one imaging unit of the pair of imaging units is first input, the timing shift amount detection unit calculates a count value indicating the phase difference of the synchronization signal between the pair of imaging units, Counting up until the synchronization signal for the other imaging means of the pair of imaging means is input, and when the synchronization signal for the other imaging means is input first, the count value is A parallax value calculation device that counts down until a synchronization signal for an image pickup means is input and outputs the counted value. A pair of imaging means mounted on the moving body provided with moving speed detecting means for detecting the moving speed of the moving body;
In disparity value calculation system including a disparity value calculation unit for calculating a disparity value between a pair of captured image data obtained by imaging the imaging area around the mobile body by the pair of imaging means,
Above for disparity value calculation unit, the disparity value calculation system characterized by using a disparity value calculation device according to any one of claims 1乃Itaru 3. The periphery of the moving object from the captured image obtained by imaging by the imaging means mounted on a movable body that moves on a moving surface, the moving surface region recognizing recognizes a moving surface area to project a moving surface to which the mobile moves In the system,
The parallax value calculation system according to claim 4,
Recognizing processing means for performing processing for recognizing the moving surface area based on a parallax value calculated by the parallax value calculating system and a luminance image obtained by at least one of the pair of imaging means. Moving surface area recognition system. Disparity value between mobile pair of a pair of captured image data obtained by imaging the imaging area around the mobile body by the imaging means to be mounted on the mobile body equipped with a moving speed detecting means for detecting a moving speed of the In a parallax value calculation method including a parallax value calculation step of calculating
A timing shift amount detecting step for detecting a shift amount of the image pickup timing between the pair of image pickup means using information indicating each image pickup timing of the pair of image pickup means input from the pair of image pickup means;
When the amount of deviation detected in the timing deviation amount detection step exceeds a predetermined deviation allowable range, execution of a use restriction process for executing processing for restricting the use of the parallax value calculated in the parallax value calculation step A process,
The use restriction process execution step sets the predetermined deviation allowable range based on the speed detection result detected by the moving speed detection means so that the predetermined deviation allowable range is widened as the moving speed of the moving body is slow. A parallax value calculation method characterized by changing. Disparity value between mobile pair of a pair of captured image data obtained by imaging the imaging area around the mobile body by the imaging means to be mounted on the mobile body equipped with a moving speed detecting means for detecting a moving speed of the In a parallax value calculation program for causing a computer to execute a parallax value calculation step for calculating
A timing shift amount detecting step for detecting a shift amount of the image pickup timing between the pair of image pickup means using information indicating each image pickup timing of the pair of image pickup means input from the pair of image pickup means;
When the amount of deviation detected in the timing deviation amount detection step exceeds a predetermined deviation allowable range, execution of a use restriction process for executing processing for restricting the use of the parallax value calculated in the parallax value calculation step And causing the computer to execute a process,
The use restriction process execution step sets the predetermined deviation allowable range based on the speed detection result detected by the moving speed detection means so that the predetermined deviation allowable range is widened as the moving speed of the moving body is slow. A program for calculating a parallax value, characterized by being changed.