JP6209825B2 - Parallax detection device and parallax detection method - Google Patents

Description

  The present invention relates to a parallax detection device that detects parallax of a pair of images photographed by a stereo camera.

  An obstacle detection apparatus that detects an obstacle such as a pedestrian or a vehicle using a stereo camera has been put into practical use. The obstacle detection device performs template matching on a plurality of images taken at the same time, and calculates a positional shift (parallax) of the same object in the plurality of images. When the parallax is calculated, the position of the object in the real space can be calculated by applying the calculated parallax to a known conversion formula. The obstacle detection device is used as a monitoring system or an in-vehicle system, and various uses are being studied.

  As processing included in the stereo image processing, there is processing for obtaining a distance by applying a triangulation technique to a pair of captured images captured by a plurality of cameras installed at intervals. In stereo image processing, a pair of imaging means (cameras) and a stereo image processing LSI that applies triangulation processing to a pair of captured images output by these imaging means are generally used. is there.

  The stereo image processing LSI realizes a triangulation process by performing a process of superimposing pixel information included in a pair of captured images to obtain a shift amount (parallax) of the coincident positions of the two images.

The measurement distance Z calculated by the triangulation process is calculated from the parallax d, the focal length f, and the base line length (interval between a plurality of cameras) b.
Z = b · f / d
Is calculated as As is clear from this equation, the distance Z increases (distant) as the parallax d decreases.
In order to increase the accuracy of the distance, the resolution of the parallax may be increased using a high-resolution image sensor, but if a high-resolution image sensor is used, the load on the stereo image processing LSI increases. For this reason, in order to maintain the processing performance of the stereo image processing LSI, it is necessary to increase the scale such as paralleling the circuits, resulting in an increase in cost.

  As a technique for reducing the load on the stereo image processing LSI, a method of limiting the area for calculating the parallax according to the traveling environment is considered (for example, see Patent Document 1). In Patent Document 1, any one of vehicle forward / backward, steering angle, vehicle speed, presence / absence of obstacles, vehicle rotation speed, vehicle turning radius, day / night, road shape, highway driving and general road driving A stereo camera device that determines an image area to be processed in accordance with a traveling environment is disclosed.

  However, the stereo camera device disclosed in Patent Literature 1 has a problem that it cannot be detected when an obstacle exists outside the parallax calculation area determined according to the traveling environment. For example, even when the steering wheel angle is steered to the left, it is difficult to detect an obstacle when the obstacle is approaching from the right.

  In view of the above problems, an object of the present invention is to provide a parallax detection device capable of reducing the load of stereo image processing while accurately detecting an object.

The present invention is a parallax detection device that detects parallax of a pair of images taken by a stereo camera, and includes a first imaging unit that takes a first image and a second image that takes a second image. A stereo camera that is spaced apart from the photographing unit, an edge image creating unit that creates an edge image by detecting an edge from the first image, and the second corresponding to an edge portion of the first image Parallax calculation means for moving the second image in the horizontal direction starting from the pixel position of the first image and calculating as the parallax information the pixel position having the maximum similarity to the first image, and moving speed detection based on the moving speed of the parallax detection device detected by means, the parallax calculating unit accepts the matching area setting means for setting a calculation target region to calculate a similarity, the set frame rate of the stereo camera from the outside A frame rate setting reception unit; and an area size instruction information table in which size information of the calculation target region is registered in association with a frame rate, and the matching region setting unit is configured to respond to the frame rate. The calculation target area of the area information read from the area area instruction information table is set at a location corresponding to the moving speed .

  It is possible to provide a parallax detection device capable of reducing the load of stereo image processing while accurately detecting an object.

It is an example of the figure explaining the schematic characteristic of the obstacle detection apparatus of this embodiment. It is an example of the figure explaining the example of mounting of the stereo camera in a vehicle. It is a figure which shows an example of the hardware block diagram of an obstruction detection apparatus. It is an example of the figure explaining the matching process by a parallax calculating part. It is an example of the figure explaining the setting of a matching area | region. It is an example of the figure explaining the width (width) of a matching area | region. It is an example of the figure explaining the dynamic determination of the width | variety of a matching area | region. It is an example of the figure explaining determination of the width | variety of the matching area | region according to a vehicle speed. It is an example of the figure which illustrates typically the relation between a vehicle speed and a matching field. It is an example of the flowchart figure which shows the procedure in which an obstacle detection apparatus calculates parallax. (Example 2) which is an example of the flowchart figure which shows the procedure in which an obstacle detection apparatus calculates parallax. It is an example of the figure which shows a response | compatibility with the coefficient k and a frame rate. (Example 3) which is an example of the flowchart figure which shows the procedure in which an obstacle detection apparatus calculates parallax. It is an example of the hardware block diagram of an obstruction detection apparatus (Example 4). (Example 4) which is an example of the flowchart figure which shows the procedure in which an obstacle detection apparatus calculates parallax. It is a figure which shows an example of the matching area | region setting screen displayed on the vehicle-mounted display apparatus.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. However, the technical scope of the present invention is not limited to this embodiment.

  FIG. 1 is an example of a diagram illustrating schematic features of the obstacle detection apparatus according to the present embodiment. There is an obstacle (pedestrian in the figure) in front of the host vehicle. The vehicle's stereo camera is shooting obstacles. The obstacle detection device performs processing on the image to create a texture image (for example, an edge image), and detects a pixel position where the obstacle exists.

  The vehicle in FIG. 1A is traveling at a low speed. In a low-speed vehicle, the moving speed of the obstacle is higher than the vehicle speed. There is a possibility of approaching at a low speed near obstacles as viewed from the own vehicle, and at a low speed, there is a possibility that the obstacles suddenly appear (approach) in the shooting range of the stereo camera. Therefore, in the obstacle detection device of this embodiment, when the vehicle speed is low, pixels in the vicinity area (shaded area in the figure; area far from the texture object) near the vehicle are used as a parallax detection area (hereinafter referred to as a matching area). decide.

  The vehicle shown in FIG. 1B is traveling at a high speed. In a high-speed vehicle, the moving speed of the obstacle is smaller than the vehicle speed. In such a situation, since the relative speed with the detected obstacle is large and approaches a distant obstacle in a short time, the vehicle to be detected is an obstacle in a far region. Therefore, when the vehicle speed is high, the obstacle detection apparatus according to the present embodiment determines a pixel in a distant area where the vehicle is far (shaded area in the drawing, an area close to the texture object) as a matching area.

  As described above, since the distance Z increases as the parallax d decreases, if the obstacle is far away, the distance on the pixel of the object corresponding to the texture object in the pair of image data should be short. . Conversely, if there is an obstacle in the vicinity, the distance on the pixel of the object corresponding to the texture object in the pair of image data should be long. Therefore, the obstacle to be detected can be efficiently detected by setting the matching area as shown in FIGS.

  Therefore, whether the vehicle speed is low or high, the matching area may be a part of the image data, so that an increase in processing load can be suppressed even if the resolution of the stereo camera is increased.

[Configuration example]
FIG. 2 is an example of a diagram illustrating an example of mounting a stereo camera in a vehicle. The stereo camera is mounted so that, for example, the front, rear, left and right sides of the host vehicle can be photographed. A stereo camera that captures the front of the host vehicle is, for example, disposed on the front side of a rearview mirror with the optical axis facing slightly forward of the vehicle. A stereo camera that captures the rear of the host vehicle is disposed, for example, in a concave portion of the back door with the optical axis facing the rear of the vehicle slightly downward. A stereo camera that shoots the side is disposed, for example, on a pillar or a roof panel with the optical axis directed slightly toward the side of the vehicle.

  The arrangement position is merely an example, and it may be mounted in a door mirror or bumper, and is not limited to the illustrated mounting location. In this embodiment, a stereo camera that mainly captures the front of the vehicle will be described.

  In the figure, the stereo camera is mounted on a bicycle or a motorcycle although it is mounted on a vehicle. Further, the present invention can be suitably applied to an object that is moved by power, such as a truck, a bus, and a one box, instead of a normal passenger car.

  The obstacle detection device measures, for example, a distance from a pedestrian in front of the vehicle, a distance from a preceding vehicle, a distance from other features (a sign, a traffic light, a power pole, a guardrail, etc.) and an obstacle. Further, image processing (for example, white line recognition, preceding vehicle recognition, pedestrian recognition, etc.) other than the processing for detecting the distance to the object may be performed using the image data of the stereo camera. The obstacle detection device can also measure the distance to an object other than the obstacle. Below, a pedestrian is mainly demonstrated as an example for a target object.

  The obstacle detection device detects a distance from the object, a relative speed, and a position (a lateral position based on the center in the vehicle width direction). These pieces of information are transmitted to a support device (not shown), and the support device performs support such as driving support, parking support, and door opening / closing support. Driving assistance calculates TTC (Time To Collision) with objects such as preceding vehicles, detects warning and automatic braking, detects driving lane markings (white line) and shoulders, and detects vehicles from the driving lane. Support to prevent deviations, etc. Parking assistance corresponds to assistance for detecting the distance from the door to an object (a wall or a side surface of the vehicle), the inclination of the surface, the direction of the surface, etc. after the vehicle stops, and prompting parking again. The door opening / closing support corresponds to the determination of whether or not the door may be opened after the vehicle is parked, and the assistance to limit the opening degree of the door that can be opened.

  FIG. 3 is a diagram illustrating an example of a hardware configuration diagram of the obstacle detection apparatus 100. A stereo camera 20, a CPU 41, a memory 42, a vehicle speed sensor 43, and a surrounding situation sensor 44 are connected to the stereo image processing LSI 30.

  The stereo camera 20 includes a reference image acquisition camera 21 and a comparative image acquisition camera 22. The reference image acquisition camera 21 and the comparative image acquisition camera 22 include an imaging device such as a CMOS or a CCD, an optical system, and a control unit that controls a gain, a shutter speed, and the like. The resolution (number of pixels) of the image sensor is not particularly limited, and examples thereof include VGA, SVGA, HD, and full HD. Although the stereo camera 20 is described as outputting a digital signal, the stereo image processing LSI 30 may output an analog signal and convert it into digital.

  The reference image acquisition camera 21 and the comparative image acquisition camera 22 are preferably cameras having the same structure. This is because if the structure is the same, less adjustment work is required to prevent a difference other than parallax from occurring in a pair of images. The names are different because the stereo image processing LSI 30 calculates the parallax while shifting the image captured by the comparative image acquisition camera 22 with reference to the image captured by the reference image acquisition camera 21. As shown in the figure, the reference image acquisition camera 21 is arranged on the left side with respect to the shooting direction, and the comparison image acquisition camera 22 is also arranged on the right side. Note that the stereo camera 20 may have three or more cameras.

  The reference image acquisition camera 21 outputs image data to the reference data input unit 311 every predetermined cycle. Hereinafter, the image data captured by the reference image acquisition camera 21 may be referred to as a “reference image”. Thereafter, the distortion correction unit 32 corrects the distortion generated by the lens and outputs the reference image to the texture detection unit 33 and the memory controller 34. Similarly, the comparison image acquisition camera 22 outputs the image data to the comparison data input unit 312 at the same timing as the reference image acquisition camera 21 (that is, at the same cycle). Hereinafter, the image data captured by the comparative image acquisition camera 22 may be referred to as a “comparison image”. The distortion correction unit 36 corrects distortion generated by the lens and outputs a comparison image to the memory controller 34.

  The distortion correction will be described. The stereo camera 20 is calibrated by photographing a calibration subject (for example, a checkered chart) at the time of shipment or after mounting on the vehicle. That is, a geometric conversion LUT (Look Up Table) is generated for converting image data so that hardware internal error factors such as camera lens distortion, optical axis deviation, focal length deviation, and image sensor distortion are minimized. ing. Alternatively, a polynomial for converting image data is created. The LUT and the polynomial are stored in a storage element accessible by the distortion correction units 32 and 26. The distortion correction units 32 and 36 correct distortion of a lens or the like using an LUT or a polynomial.

  The texture detection unit 33 performs texture detection on the image data of the reference image acquisition camera 21 to detect a texture target. Thereby, texture data (image data) is created. The texture data is data indicating the position of the contour of the object present in the reference image. For example, edge detection processing is performed on the reference image, and pixels having edge strengths equal to or higher than a predetermined level are detected as the position of the contour of the object. The texture detection may be performed on the comparative image acquisition camera 22. In this case, the reference image and the comparison image are interchanged.

  The memory controller 34 stores the reference image, comparison image, and texture data in the memory 42. The memory 42 is a high-speed volatile memory such as DRAM or SDRAM.

  As described above, the memory 42 periodically stores a set of reference images, comparison images, and texture data. When the accumulation of one set of reference image, comparison image, and texture data is completed, the memory controller 34 reads three data from the memory 42 and transfers them to the parallax calculation unit 35.

  The matching start position setting unit 38 determines an appropriate matching region based on the vehicle speed input from the vehicle speed sensor 43 and the vehicle status input from the vehicle status sensor, and sets the matching region to the parallax calculation unit 35. Details will be described later.

  The parallax calculation unit 35 specifies the pixel for performing parallax detection from the reference image based on the texture data transferred from the memory controller 34, performs the parallax calculation from the matching region of the comparison image set by the matching start position setting unit 38, Parallax is calculated.

  Note that the CPU 41 may be mounted on a microcomputer disposed on the same substrate as the stereo image processing LSI 30 or may be mounted on a microcomputer disposed on a different substrate. The CPU 41 executes a predetermined program and controls the stereo image processing LSI 30. Specifically, the frame rate, resolution, brightness, etc. are set when the stereo image processing LSI 30 is activated.

  The vehicle speed sensor 43 is disposed on each wheel of the vehicle, and is a sensor that outputs a pulse proportional to the rotational speed of the wheel, for example. In addition, the surrounding situation sensor 44 is used for the brightness of the surroundings, the weather, the type of road in which the vehicle is running (highway driving or general road driving), the parking lot (slowing is required), It is a sensor that detects surrounding conditions such as waiting for a signal, crossing a pedestrian crossing, passing an intersection with poor visibility, or turning right at an intersection with an oncoming vehicle. This sensor does not have to be a single sensor, and it is only necessary to detect the surrounding situation by combining detection signals of various sensors mounted on the vehicle.

  For example, ambient brightness is detected by an illuminance sensor for shutter speed or conlight (auto light). Of the weather, rain is detected by the operation of the wiper and the rain sensor, and fog is detected from the lighting of the fog lamp. The traveling road type (express road traveling or general road traveling) and traveling in a parking lot are detected from a position detection device using a GNSS (Global Navigation Satellite System) or the like and a road map. Waiting for a signal at a pedestrian crossing is detected so that the vehicle speed can be regarded as zero or zero, and a red display signal is photographed by a camera or the like. The fact that a pedestrian crosses a pedestrian crossing is detected because a pedestrian crossing the front is recognized by a camera or the like. Passing an intersection with poor visibility is detected from a position detection device using GNSS or the like, a road map, a vehicle speed, and the like. Turning right at an intersection with an oncoming vehicle is detected from a position detection device using GNSS or the like, a road map, an operating direction of a winker, and the like.

[Parallax calculation]
FIG. 4 is an example of a diagram illustrating the matching process performed by the parallax calculation unit 35. 4A shows an example of a reference image, FIG. 4B shows an example of a comparative image, and FIG. 4C shows an example of texture data. In both cases, the pedestrian is photographed, but the photographing position of the pedestrian in the horizontal direction differs between the reference image and the comparative image due to the parallax. Moreover, the pixel position where the pedestrian is photographed in the reference image can be specified (or estimated) by the texture data.

  First, for the sake of explanation, a conventional matching process will be described.

  (i) As shown in FIG. 4D, the parallax calculation unit 35 searches for a pixel in the horizontal direction or the vertical direction using, for example, the upper left vertex of the reference image as an origin, and detects a pixel determined as a texture. To do. Let this pixel be a pixel of interest, and its pixel position be (X, Y).

  (ii) The parallax calculation unit 35 applies each pixel of the reference image to a pixel near the pixel position (X, Y) of the pixel of interest (for example, a block of 5 × 5 to 11 × 11 having the pixel of interest as the center). The brightness value of each pixel of the comparison image is compared with the brightness value of the comparison image. Specifically, for each pixel of the reference image block and each pixel of the comparison image block at the same position as the reference image block, the difference in luminance value is calculated for each pixel and summed (or the square of the difference is calculated). Sum up). The smaller the sum of luminance differences, the higher the similarity (matching degree).

  (iii) The parallax calculation unit 35 moves the block of the comparison image by one pixel in the horizontal direction with the pixel position (X, Y) as the base point, and similarly calculates the sum of the luminance differences of the blocks of the reference image. Since the reference image acquisition camera 21 and the comparison image acquisition camera 22 are arranged in parallel in the horizontal direction (because differences other than parallax have been corrected), the blocks of the comparison image need only move in the horizontal direction. .

  (iv) The parallax calculation unit 35 calculates the sum of luminance differences within a predetermined horizontal range (until the end in the horizontal direction in FIG. 4D). Then, the horizontal position (amount of movement of the comparison image block in the horizontal direction) at which the degree of similarity is maximum (the total is minimum) is output as parallax d.

  In the example of FIG. 4E, the pixel position of the pixel of the comparison image corresponding to the target pixel (X, Y) of the reference image is (X ′, Y) in the block of the comparison image having the maximum similarity. Suppose there is. In this case, the parallax d is “d = X′−X”.

  The parallax calculator 35 calculates the parallax d for the texture object of the texture data. Since the parallax d is accurately the number of pixels, the parallax d is converted into a dimension of distance using the pixel pitch.

  Since the above description is a conventional example, the matching region is defined from the pixel position determined as the texture to the last pixel in the horizontal direction. Therefore, when the image data has a high resolution and the number of pixels increases, the processing load of the stereo image processing LSI 30 increases.

  Therefore, the parallax calculation unit 35 of the present embodiment performs matching processing on the matching region set by the matching start position setting unit 38.

[Setting of matching area]
FIG. 5 is an example of a diagram illustrating the setting of the matching area. The matching start position setting unit 38 of the present embodiment determines a matching area according to the vehicle speed.

  FIG. 5A shows an example of the matching area when the vehicle speed is equal to or higher than the threshold value. When the vehicle speed is equal to or higher than the threshold value, the matching start position setting unit 38 sets the matching region so as to search only the pixels close to the texture object in the comparison image. Pixels close to the texture object refer to a predetermined number of pixel ranges from the technique “from texture object” or “from texture object + α (several pixels to several tens of pixels)”. In FIG. 5, since the reference image is taken by the left camera, the matching area exists in the right direction from the texture object. Therefore, a predetermined number of pixel ranges are matched from the texture object or its vicinity to the right direction. It is an area. When the reference image is taken by the right camera, the location of the matching area is reversed.

  Note that the threshold value is a vehicle speed at which the vehicle can be regarded as traveling slowly, and is, for example, 20 [km / h] to 30 [km / h].

  FIG. 5B shows an example of the matching area when the vehicle speed is less than the threshold value. When the vehicle speed is less than the threshold value, the matching start position setting unit 38 sets the matching area so as to search for a pixel area far from the texture object of the comparison image. In the figure, the periphery (right end) of the comparison image is a matching region as a far pixel region.

  “Far” means that the pixel region far from the texture object is set as the matching region, at least when the vehicle speed is equal to or higher than the threshold value. How far away from the texture object can be optimized depending on how far away the object is detected when the vehicle speed is high. Note that FIG. 5 (b) emphasizes that it is “far” from the textured object.

  In FIG. 5B, since the reference image is captured by the left camera, the matching region exists in the right direction with respect to the texture object. For this reason, a predetermined number of pixel ranges close to the right end or the right end in the horizontal direction are matching regions. When the reference image is taken by the right camera, the matching area is located in the left direction with respect to the texture object. Therefore, a predetermined number of pixel ranges close to the left or right edge in the horizontal direction are the matching areas.

  As shown in FIG. 5 (a), when the vehicle speed is fast, it is easy to detect the distance and relative speed with a pedestrian approaching from a distance by setting a matching area near the pixel position determined to be texture. Become. Further, as shown in FIG. 5B, when the vehicle speed is low, by setting a matching region at a position far from the pixel position determined as the texture, it is easy to detect an object entering from the vicinity or the periphery. In any case, in the past, all the pixels in the right direction from the texture object have been searched, but in the present embodiment, some pixels in the right direction from the texture object are set in the matching region. Therefore, the obstacle detection apparatus 100 of the present embodiment can reduce the amount of calculation for matching.

  In this embodiment, the matching area is not limited to that shown in FIGS. 5A and 5B, but the “start position” of the matching area may be set. That is, when the similarity does not exceed the threshold value in the set matching area, the matching process is performed on the area other than the matching area. In this way, the parallax can be calculated in a short time, and finally the parallax can be obtained even if the matching region is not appropriate.

[Size of matching area]
FIG. 6 is an example of a diagram for explaining the width (width) of the matching region. In FIG. 5, the width of the matching area is the same when the vehicle speed is low and when the vehicle speed is high. For example, as shown in FIG. 6A, the width of the matching area when the vehicle speed is slow and when the vehicle speed is fast is both a (unit is the number of pixels).

  Further, as shown in FIG. 6 (b), the matching regions when the vehicle speed is low and when the vehicle speed is high may partially overlap. In FIG. 6B, since the width of the matching area is long, in the area on the right side of the texture object, the matching areas when the vehicle speed is low and when the vehicle speed is high partially overlap.

Further, the width of the matching area does not have to be the same when the vehicle speed is low and when the vehicle speed is high.
FIG. 6C shows an example in which the width of the matching area is wider when the vehicle speed is slow, and FIG. 6D shows an example in which the width of the matching area is narrower when the vehicle speed is slow. . As shown in FIG. 6C, when the vehicle speed is low, the object that enters from the vicinity or the vicinity can be easily detected by widening the matching area. Further, as shown in FIG. 6D, by widening the matching area when the vehicle speed is fast, it is easy to capture and monitor the relative distance and the like even if a distant pedestrian moves.

  Further, as shown in FIG. 7, the width of the matching area may be determined dynamically. FIG. 7A is an example for explaining dynamic determination of the width of the matching region. The matching start position setting unit 38 measures the length L from the texture object to the last pixel in the horizontal direction. Then, half of the length L is determined as the width (L / 2) of the matching region. By doing so, when the case where the vehicle speed is fast and the case where the vehicle speed is slow are combined, it is possible to search from the texture object to the end in the horizontal direction without deficiency and duplication. It may be 2/3 of L instead of half of the length L.

  Alternatively, the low speed region below the threshold and the high speed region above the threshold may be divided into about three stages, respectively, and the width of the matching region may be determined according to the vehicle speed.

  FIG. 8 shows an example of the matching region when the low-speed region is divided into three stages of VL2 to VL3, VL1 to VL2, and VL1 (VL1 <VL2 <VL3 <threshold). Since the region is a low speed region, the region far from the texture object is the matching region. However, the slower the vehicle speed, the larger the matching region. When the vehicle speed is slow, there is a high possibility that a pedestrian or the like will enter from the vicinity. Therefore, it is easy to detect an object by increasing the matching area.

  Similarly, in FIG. 8B, the high-speed region is divided into three stages of less than VH1, VH1 to less than VH2, and VH2 or more (threshold value <VH1 <VH2). Since it is a high-speed area, the area close to the texture object is the matching area, but the matching area becomes larger as the vehicle speed increases. When the vehicle speed is high, there is a high possibility that the detected object moves relatively. Therefore, it is easy to maintain capture of the object by increasing the matching area.

  In the past, the matching area was set to be discontinuous depending on whether the vehicle speed was low or high. However, the matching area may be moved continuously with the vehicle speed.

  FIG. 9 is an example of a diagram schematically illustrating the relationship between the vehicle speed and the matching area. 9 (a) is 10 [km / h], FIG. 9 (b) is 20 [km / h], FIG. 9 (c) is 30 [km / h], and FIG. [Km / h], FIG. 9 (e) shows the matching region when the speed is 50 [km / h] and FIG. 9 (f) shows the matching area when the speed is 60 [km / h]. As shown in the figure, the faster the vehicle speed, the closer the matching area is to the texture object. Therefore, in the low speed region, there is a matching region in a region away from the texture object, and in the high speed region, the matching region is in a region near the texture object. However, in FIG. 9, the location of the matching region can be changed continuously from low speed to high speed. Thereby, it is possible to optimize and match a region to be watched according to the vehicle speed (a near region at a low speed and a far region at a high speed).

  In FIG. 9, the matching area has a certain width, but it may be wider as the vehicle speed is faster, or may be wider as the vehicle speed is slower. Further, it may be widened in the low speed region and the high speed region, and narrowed in the medium speed region.

[Operation procedure]
FIG. 10 is an example of a flowchart illustrating a procedure for calculating the parallax by the obstacle detection apparatus 100 according to the present embodiment.

  The reference image acquisition camera 21 captures a reference image, and the comparative image acquisition camera 22 captures a comparison image (S10). The reference image and the comparison image are each subjected to distortion correction and stored in the memory 42 by the memory controller 34.

  The texture detection unit 33 creates texture data from the reference image (S20). The texture data is stored in the memory 42 by the memory controller 34.

  Next, the matching start position setting unit 38 acquires the vehicle speed from the vehicle speed sensor 43 (S30). And it is determined whether a vehicle speed is less than a threshold value (S40).

  When the vehicle speed is less than the threshold (Yes in S40), the matching start position setting unit 38 sets a pixel far from the texture object as a matching region (S50).

  When the vehicle speed is not less than the threshold value (No in S40), the matching start position setting unit 38 sets a pixel close to the texture object in the matching region (S60).

  The parallax calculation unit 35 determines a target pixel from the texture data, performs a matching process between the reference image and the comparison image, and calculates the parallax (S70).

  As described above, the obstacle detection apparatus 100 according to the present embodiment can reduce the load of stereo image processing while accurately detecting an object by changing the matching area between low speed traveling and high speed traveling. it can.

  In the present embodiment, the obstacle detection device 100 that sets the matching area according to the surrounding situation of the host vehicle in addition to changing the matching area according to the vehicle speed will be described.

  For example, there is a case where the host vehicle travels in a parking lot that should slow down. However, the vehicle may accelerate to a high speed without the driver's knowledge. On the other hand, nearby objects do not move much in the parking lot. Therefore, in the case of a parking lot, it may be set as a matching area at high speed regardless of the vehicle speed.

  In addition, when waiting for a signal at a pedestrian crossing or when a pedestrian is passing, there is a possibility that a pedestrian or the like may enter, so the matching area should be changed between the low speed range and the high speed range as in the first embodiment. It is.

  As described above, by considering the peripheral information, an appropriate pixel region can be determined as a matching region even when an unpredictable operation (for example, high-speed operation in a slow driving area) is performed.

  FIG. 11 is an example of a flowchart illustrating a procedure for calculating the parallax by the obstacle detection apparatus 100 according to the present embodiment. The procedure of FIG. 11 is the same as that of FIG. 10, but in step S33, the matching start position setting unit 38 specifies the surrounding situation based on the detection signal of the surrounding situation sensor 44 (S33).

  Then, the matching start position setting unit 38 determines whether or not the surrounding situation is a parking lot (S35). When the surrounding situation is a parking lot (Yes in S35), a pixel close to the texture object is set as a matching area (S60).

  When the surrounding situation is not a parking lot (No in S35), the matching start position setting unit 38 determines whether or not the vehicle speed is less than a threshold value (S40). The subsequent processing is the same as in FIG.

  In FIG. 11, pixels close to the texture object are set as the matching area when the surrounding situation is a parking lot, but pixels far from the texture object are set as the matching area in another specific surrounding situation. Also good.

  According to this embodiment, the matching area can be set according to not only the vehicle speed but also the surrounding situation.

  In this embodiment, an obstacle detection apparatus 100 that sets a matching area according to the frame rate of the stereo camera 20 will be described.

  The CPU 41 sets the frame rate in the stereo image processing LSI 30 via the CPU I / F 37. The reason for dynamically changing the frame rate is, for example, when the load is reduced because the temperature of the stereo image processing LSI 30 is increased, or when the frame rate is decreased to increase the shutter time because it is nighttime. There is a case.

  When the high frame rate is set, the stereo image processing LSI 30 has to process one frame in a short time, so that the processing load increases. Therefore, it is preferable to narrow the matching area. On the other hand, when a low frame rate is set, the processing time for one frame is sufficient, so that the matching area can be widened.

  By setting the matching area according to the frame rate, the parallax can be calculated with the highest possible accuracy at the requested frame rate.

In order to change the size of the matching area according to the frame rate, for example, the size of the reference matching area may be multiplied by a coefficient k corresponding to the frame rate.
Matching area width = k × reference matching area width FIG. 12 is an example of a diagram illustrating the correspondence between the coefficient k and the frame rate. The information in FIG. 12 is stored as a table in the stereo image processing LSI 30, for example. The information in FIG. 12 is an example of the area size instruction information table in the claims. If the size of the reference matching region is the size of the matching region with the lowest frame rate, the coefficient k is less than 1.

  Since k = 1.0 at low speed, the size of the matching area remains at the reference value when the frame rate is low.

  Since k = 0.8 at medium speed, the size of the matching area is 80% of the reference value when the frame rate is medium speed.

  Since k = 0.6 at high speed, the size of the matching area is 60% of the reference value when the frame rate is medium speed.

  The coefficient k may be determined in more stages, or a function for calculating the coefficient k for an arbitrary vehicle speed may be prepared and the coefficient k may be obtained steplessly.

  FIG. 13 is an example of a flowchart illustrating a procedure for calculating the parallax by the obstacle detection apparatus 100 according to the present embodiment.

  In this embodiment, first, the matching start position setting unit 38 determines whether or not there is a change in the frame rate (S2). For example, the CPU I / F 37 notifies the matching start position setting unit 38 that the frame rate has been changed by the CPU 41.

  When the frame rate is changed (Yes in S2), the matching start position setting unit 38 reads the coefficient k (S4).

  The processing from step S10 to S50 is the same as that in FIG. In steps S52 and S62, the matching start position setting unit 38 determines the size of the matching area by the coefficient k (S52, S62).

  Therefore, according to the obstacle detection apparatus 100 of the present embodiment, in addition to the effects of the first and second embodiments, the size of the matching area can be changed according to the frame rate, and the stereo image processing LSI 30 can be used at the frame rate as much as possible. The parallax can be calculated with high accuracy.

  In the third embodiment, the size of the matching area is determined based on the frame rate set by the CPU 41. However, in this embodiment, the obstacle detection apparatus that determines the width of the matching area based on the frame rate set by the stereo image processing LSI 30. 100 will be described.

  For example, when the reference image captured by the stereo camera 20 does not change much, a high frame rate is not required. Therefore, the stereo image processing LSI 30 calculates the required frame rate from the change rate of the reference image, and determines the size of the matching area.

For example, the case where the rate of change of the reference image is low is
A. A state where the vehicle and the surrounding objects are stopped. For example, the vehicle and surrounding objects are moving in the same direction at the same speed. The former can be assumed to be stopped due to traffic lights or traffic congestion, and the latter can be assumed to be traveling at a constant speed on an empty road.

Cases where the rate of change of the reference image is high
C. A state in which the object moves greatly. This is a situation where the vehicle is traveling on a curve or the like. It can be assumed that the former is a case where a pedestrian runs or the preceding vehicle decelerates rapidly, and the latter is a case where the vehicle is turning right or left or a sharp curve.

  FIG. 14 is a diagram illustrating an example of a hardware configuration diagram of the obstacle detection apparatus 100 according to the present embodiment. In the present embodiment, the components denoted by the same reference numerals in FIG. 3 perform the same function, and therefore, only the main components of the present embodiment may be mainly described. The stereo image processing LSI 30 according to this embodiment includes a frame rate determination unit 39.

  First, in this embodiment, the memory controller 34 stores the texture data of the previous frame in the memory 42. The frame rate determination unit 39 calculates the rate of change by comparing the texture data output from the texture detection unit 33 and the texture data of the previous frame read from the memory 42. For example, the difference between the pixel values of the two texture data is calculated for each pixel or each pixel flock, and the difference between the pixels is summed. As the total difference, a fixed value with a sufficiently large change rate is set, and the change rate can be obtained by dividing the total difference by the fixed value.

The frame rate determination unit 39 reads out the frame rate held in advance in association with the change rate and sets it in the matching start position setting unit 38. The matching start position setting unit 38 determines the coefficient k from the correspondence between the coefficient k and the frame rate in FIG.
Large change rate: 60fps frame rate
Rate of change Medium: Frame rate 50fps
Change rate small: Frame rate 40fps
In the states of Examples A and B above, since a high frame rate is not necessary, the accuracy of parallax calculation can be improved by setting the frame rate low and setting the matching region wide. In the states C and D, although the high frame rate is set, the matching area is set to be narrow, so that an increase in the load on the stereo image processing LSI 30 can be suppressed.

  Since the matching start position setting unit 38 reads the coefficient k in accordance with the frame rate, as in the third embodiment, when the high frame rate is set, the matching region is narrowed, and when the low frame rate is set. Can widen the matching area.

  FIG. 15 is an example of a flowchart illustrating a procedure for calculating the parallax by the obstacle detection apparatus 100 according to the present embodiment.

  The processing in steps S10 and S20 is the same as that in FIG. In step S22, the frame rate determination unit 39 determines the frame rate according to the change rate of the texture data (S22). The frame rate determination unit 39 sets the frame rate in the matching start position setting unit 38.

  Then, the matching start position setting unit 38 reads the coefficient k associated with the frame rate (S24).

  Thereafter, the vehicle speed is acquired, the surrounding situation is specified, and the like, and in steps S52 and S62, the matching start position setting unit 38 determines the size of the matching region by the coefficient k (S52, S62).

  According to this embodiment, since the stereo image processing LSI 30 determines the frame rate and determines the size of the matching area, the parallax can be calculated with the highest possible accuracy while suppressing the load.

  In this embodiment, an obstacle detection apparatus 100 that allows a user to arbitrarily set a matching area will be described.

  FIG. 16 is a diagram illustrating an example of a matching area setting screen displayed on a display device mounted on a vehicle. This display device is, for example, a display on which a navigation system displays a road map.

  In FIG. 16A, a vehicle speed mode button 601 and an arbitrary setting button 602 are displayed. When the user selects the vehicle speed mode button 601, the obstacle detection apparatus 100 sets a matching area according to the vehicle speed described in the first to fourth embodiments. When the user selects the arbitrary setting button 602, the screen in FIG. 16B is displayed. This screen displays “Please move the diagonal line icon to set the matching area”. The user moves the hatched icon 603 to set the matching area in a state where “low speed” and “high speed” are selected (pressed and reversed).

  In addition, since the hatched icon 603 is widened by a double tap or the like, the user can arbitrarily determine the size of the matching area.

  In addition, the setting of FIG. 16 may be set with respect to a server by connecting to a server from a terminal other than performing with an in-vehicle device. The server sets a matching area in the vehicle using a cellular phone network or the like.

  According to the present embodiment, in addition to the effects of the first to fourth embodiments, the user can set an arbitrary matching region.

20 Stereo camera 30 Stereo image processing LSI
32, 36 Distortion correction unit 33 Texture detection unit 34 Memory controller 35 Parallax calculation unit 38 Matching start position setting unit 41 CPU
42 Memory 100 Obstacle Detection Device 311 Reference Data Input Unit 312 Comparison Data Input Unit

JP 2009-146217 JP

Claims (8)

A parallax detection device that detects parallax of a pair of images captured by a stereo camera,
A stereo camera in which a first photographing means for photographing a first image and a second photographing means for photographing a second image are arranged apart from each other;
Edge image creation means for creating an edge image by detecting an edge from the first image;
A pixel determined as an edge of the first image is a pixel of interest, a neighboring image centered on the pixel of interest is a first pixel block, and a second image at the same position as the block of the first pixel Parallax calculation means for moving the block in the horizontal direction and calculating, as parallax information, a pixel position at which the degree of similarity with the block of the first image is maximized;
A matching region setting unit that sets a calculation target region in which the parallax calculation unit calculates a similarity based on the moving speed of the parallax detection device detected by the moving speed detection unit;
Frame rate setting accepting means for accepting the setting of the frame rate of the stereo camera from the outside;
An area size instruction information table in which the size information of the calculation target area is registered in association with a frame rate,
The matching area setting means sets the calculation target area of the area information read from the area area instruction information table according to the frame rate to a place corresponding to the moving speed. Detection device. The matching area setting means acquires a surrounding situation of a moving body on which the parallax detection device is mounted from a surrounding situation detection means, and sets the calculation target area according to the surrounding situation.
The parallax detection device according to claim 1. A frame rate determining means for determining a frame rate by monitoring a rate of change of an image captured by the stereo camera;
An area size instruction information table in which the size information of the calculation target area is registered in association with a frame rate,
The matching area setting unit sets the calculation target area of the area information read from the area area instruction information table in a place corresponding to the moving speed according to the frame rate determined by the frame rate determining unit. ,
The parallax detection device according to claim 1 . A calculation target area setting receiving unit that receives a setting of a location and a size of the calculation target area corresponding to the moving speed;
The matching area setting means sets the calculation target area of the received area at the place where the calculation target area setting reception means has received the setting.
The parallax detection device according to claim 1, wherein When the moving speed is less than the threshold, the matching area setting unit is farther than when the moving speed is equal to or greater than the threshold with reference to the pixel position of the second image corresponding to the edge portion of the first image. Set the calculation target area to
The parallax detection device according to claim 1, wherein: The parallax detection device according to claim 1, wherein the matching area setting unit changes a size of the calculation target area according to the moving speed. The matching area setting means increases the area of the calculation target area as the moving speed is high, and decreases the area of the calculation target area as the moving speed is low.
The parallax detection device according to claim 6 . A parallax detection method for detecting parallax of a pair of images taken by a stereo camera,
A step of taking a first image by a first photographing unit of the stereo camera, and a step of taking a second image by a second photographing unit of the stereo camera;
An edge image creating means creating an edge image by detecting an edge from the first image;
A step matching area setting means, which on the basis of the moving speed of the detected visual difference detecting device by the moving speed detecting means, sets the calculated target region parallax calculating means for calculating a degree of similarity,
The parallax calculation means sets the pixel determined as the edge of the first image as the target pixel, sets the neighboring image centered on the target pixel as the first pixel block, and the same position as the block of the first pixel Moving the second image block in the horizontal direction to calculate a pixel position having the maximum similarity with the first image block as disparity information;
A step of receiving a frame rate setting of the stereo camera from the outside;
The matching area setting means reads from the area size instruction information table according to the frame rate with reference to the area size instruction information table in which the area information of the calculation target area is registered in association with the frame rate. Setting the area to be calculated of the area information at a location corresponding to the moving speed;
A parallax detection device comprising:

Images