JP7202123B2 - Vehicle stereo camera device - Google Patents

Description

The present invention relates to a vehicle stereo camera device that selects two cameras from a plurality of cameras to obtain a stereo image.

A stereo camera equipped with a pair of left and right stereo cameras on the vehicle that captures images of the driving environment in front of the vehicle, thereby recognizing preceding vehicles and various obstacles and measuring the distance between the vehicle and the target. Camera devices are known.

Further, in Patent Document 1 (Japanese Patent Application Laid-Open No. 11-39596), a vehicle is equipped with a long-distance stereo camera and a short-distance stereo camera, and based on the driving environment information acquired by both stereo cameras, a near-field image is constantly detected. A technique has been disclosed for recognizing an object in a wide range from a long distance to a long distance and determining the distance to the own vehicle.

In addition, in Patent Document 2 (Japanese Patent Application Laid-Open No. 2013-61317), three cameras are arranged at a predetermined interval, and two cameras with the shortest baseline length are selected at low speed, and at medium speed discloses a technique for obtaining a stereo image by selecting two cameras having intermediate baseline lengths and selecting two cameras having the longest baseline lengths at high speed.

JP-A-11-39596 JP 2013-61317 A

By the way, the driving environment of a vehicle is not only fine weather, but also bad weather such as rain and snow, and raindrops, snow, etc. adhere to the windshield. In addition, even if the weather is fine, the windshield may become dirty due to splashes of mud, bird droppings, and the like. As described in each of the above-mentioned patent documents, each camera is arranged on the vehicle interior side of the windshield, and if the view of the camera is obstructed by dirt on the windshield, the image obtained by the camera will be corrupted. . Also, if the camera lens is dirty, the image will be corrupted because the field of vision will be obstructed. In addition, image defects can also occur due to malfunctions in the camera itself.

When a stereo image is acquired by a pair of cameras, if the image of one camera fails, the image of the other camera is acquired, and even if the object can be recognized, the distance to the object cannot be achieved. It becomes impossible to acquire a stereo image from which distance can be measured.

As a result, based on the stereo image, when performing driving support control such as well-known following distance control (ACC: Adaptive Cruise Control) and lane keeping control (ALK: Active Lane Keep), one camera Driving support is canceled due to image failure. Also, in automatic driving, if a redundant system is configured with a map locator and an in-vehicle camera, redundancy cannot be ensured due to a defect in the image of one of the cameras, so automatic driving is canceled.

Immediate cancellation of driving support control or automatic driving due to the failure of one camera not only makes the driver panic, but also gives the driver a sense of discomfort.

For example, as disclosed in Patent Document 1, a long-distance stereo camera and a short-distance stereo camera are provided. It is also conceivable to substitute a stereo camera that has failed with a stereo camera.

However, when the function of the short-range stereo camera is substituted by the long-range stereo camera, it is not possible to adequately cope with running at low and medium speeds. Conversely, if the function of the long-distance stereo camera is substituted with the short-distance stereo camera, there is a problem that it is not possible to sufficiently cope with high-speed running.

In view of the above circumstances, the present invention provides a method for driving support control and automatic driving without significantly impairing the driving state even if at least one of the four cameras fails, even if it is substituted with another camera. To provide a vehicular stereo camera device capable of continuing driving or allowing a driver to smoothly take over the driving, and capable of reducing discomfort given to the driver.

The present invention includes a driving information acquiring means for acquiring driving information of the own vehicle , a pair of cameras for at least a long distance and a short distance , which are arranged on the own vehicle at a predetermined base line length and capture an image of the outside of the own vehicle. and one of the pair of long-distance cameras and the pair of short-distance cameras of the imaging means based on the travel information acquired by the travel information acquisition device. and acquires a stereo image of the imaging area by the selected pair of cameras, and the pair of long-distance cameras and the pair of short-distance cameras selected by the camera image selection means and driving environment recognition means for recognizing the driving environment outside the vehicle based on a stereo image from one of the pair of cameras, wherein the camera image selection means selects from the selected pair of cameras camera failure determination means for performing stereo matching processing on the acquired stereo image to check whether the image is defective; and if the camera failure determination means determines that the image is defective, the selected checks whether it is the pair of long-distance cameras or the pair of short-distance cameras, and if it is determined that one of the pair of long-distance cameras is defective, A substitute camera is selected from cameras other than the selected pair of cameras, a stereo image of a substitute imaging area is acquired by the substitute camera and a normal camera in the pair of cameras, and the pair of cameras for close range is acquired. a substitute image obtaining means for obtaining, as a monocular image, an image captured by a normal camera of the pair of short-range cameras when it is determined that one of the cameras is defective .

According to the present invention, at least one of a pair of long-distance cameras and a pair of short-distance cameras is selected, and stereo matching processing is performed on stereo images acquired from the selected pair of cameras to correct image defects. If it is determined that the image is defective, it is checked whether the long-range camera pair or the short-range camera pair has been selected. If one of the cameras is determined to be defective, a substitute camera for the malfunctioning camera is selected from cameras other than the selected pair of cameras, and this substitute camera is substituted by a normal camera in the pair of cameras. Since stereo images of the imaging area are acquired, even if a camera that has failed is replaced with another camera, the driving state is not significantly impaired, and driving support control, automatic driving can be continued, or driving can be performed. It is possible to allow the driver to take over the driving smoothly. As a result, the discomfort given to the driver can be reduced.

Functional block diagram showing the configuration of an automatic driving support system equipped with a vehicle stereo camera device Front view of a vehicle equipped with a stereo camera device Flowchart showing camera image selection processing routine (part 1) Flowchart showing camera image selection processing routine (Part 2) Bird's-eye view showing acquisition area of stereo image Explanatory diagram showing acquisition areas of stereo images by a combination of four cameras Explanatory drawing showing a state in which a stereo image of a long-distance area is acquired using cameras at both ends. Explanatory diagram showing a state in which a stereo image of a middle-distance left region is acquired using the camera on the left end and the camera on the right side of the center. Explanatory diagram showing a state in which a stereo image of the middle-range right region is acquired using the camera on the right end and the camera on the left side of the center. Explanatory diagram showing a state in which a stereo image of the near-field left region is acquired using the leftmost camera and the center left camera. Explanatory diagram showing a state in which a stereo image of the near-field central region is acquired using the cameras on the left and right of the center. Explanatory diagram showing a state in which a stereo image of the near-field right region is acquired using the rightmost camera and the center right camera. A diagram showing the combination of the front recognition area for acquiring stereo images and the camera to be used A chart showing combinations of forward recognition areas and alternative monocular cameras to be substituted when a specific camera fails Graph showing the relationship between the imaging area of the defective stereo image and the imaging area of the stereo image acquired to identify the defective camera

An embodiment of the present invention will be described below based on the drawings. The automatic driving support system shown in FIG. 1 is installed in the host vehicle M (see FIG. 2). This automatic driving support system 1 has a locator unit 11 for detecting the position of the vehicle and a stereo camera device 21 for recognizing the driving environment in front of the vehicle. If one of the locator unit 11 and the stereo camera device 21 malfunctions, a redundant system is constructed in which the other unit temporarily continues automatic driving assistance. In addition, the automatic driving support system 1 constantly monitors whether the shape of the road on which the vehicle is currently traveling is the same between the locator unit 11 and the stereo camera device 21, and if they are the same, the automatic driving support is continued.

The locator unit 11 estimates the position of the own vehicle M on the road map (own vehicle position) and acquires road map data ahead of the own vehicle position. On the other hand, the stereo camera device 21 obtains the road curvature at the center of the demarcation line that divides the left and right lanes of the vehicle M, and the lateral position deviation of the vehicle M in the vehicle width direction with the center of the left and right demarcation line as a reference. to detect Furthermore, the stereo camera device 21 can detect three-dimensional objects including a preceding vehicle in front of the own vehicle M, pedestrians and two-wheeled vehicles (bicycles, motorcycles) trying to cross in front of the vehicle M, and signal appearance (lighting color). , road signs, etc.

The locator unit 11 has a map locator calculator 12 and a high precision road map database 16 as storage means. The map locator calculation unit 12, the forward running environment recognition unit 25, and the automatic driving control unit 26, which will be described later, are composed of a well-known microcomputer equipped with a CPU, RAM, ROM, nonvolatile memory, etc., and peripheral devices thereof. In the ROM, programs to be executed by the CPU and fixed data such as data tables are stored in advance.

A GNSS (Global Navigation Satellite System) receiver 13 and an autonomous travel sensor 14 are connected to the input side of the map locator calculation unit 12 . The GNSS receiver 13 receives positioning signals transmitted from a plurality of positioning satellites. In addition, the autonomous traveling sensor 14 enables autonomous traveling in an environment where the reception sensitivity from the GNSS satellite is low and the positioning signal cannot be effectively received, such as traveling in a tunnel. It consists of an acceleration sensor and the like. That is, the map locator calculation unit 12 performs localization based on the movement distance and direction based on the vehicle speed detected by the vehicle speed sensor, the yaw rate (yaw angular velocity) detected by the yaw rate sensor, and the longitudinal acceleration detected by the longitudinal acceleration sensor.

The map locator calculation unit 12 has a function of estimating the position of the vehicle, which is a vehicle position estimation calculation unit 12a. A map information acquisition unit 12b that acquires road map information including environmental information, and a target travel route setting calculation unit 12c that sets a target travel route (target travel route) of the own vehicle M are provided.

The high-precision road map database 16 is a large-capacity storage medium such as an HDD, and stores high-precision well-known road map information (local dynamic map). This high-precision road map information has a hierarchical structure in which additional map information necessary to support automatic driving is superimposed on the static information layer of the lowest layer as a base.

The map information acquisition unit 12b described above acquires the current location and road map information ahead from the road map information stored in the high-precision road map database 16. FIG. This road map information includes surrounding environment information. This surrounding environment information includes not only static location information such as road types (general roads, expressways, etc.), road shapes, left and right division lines, road signs, stop lines, intersections, and traffic lights, but also congestion information and accident information. Alternatively, dynamic position information such as traffic restrictions due to construction work is also included.

Then, for example, based on the destination set by the driver during automatic driving, the route map information from the vehicle position (current location) estimated by the vehicle position estimation calculation unit 12a to the destination is obtained from the road map information. Then, the acquired route map information (lane data on the route map and its surrounding information) is transmitted to the own vehicle position estimation calculation unit 12a.

The own vehicle position estimation calculation unit 12a acquires the position coordinates of the own vehicle M based on the positioning signal received by the GNSS receiver 13, map-matches the position coordinates on the route map information, and displays the position coordinates of the own vehicle on the road map. The position (current location) is estimated, the driving lane is specified, and the road shape of the driving lane stored in the route map information is acquired and stored sequentially.

Furthermore, the vehicle position estimation calculation unit 12a switches to autonomous navigation in an environment in which it is not possible to receive an effective positioning signal from the positioning satellite due to a decrease in the sensitivity of the GNSS receiver 13, such as when traveling in a tunnel. Localization is performed by the sensor 14 .

The target travel route setting calculation unit 12c first sets a target travel route for automatically driving the own vehicle M along the lane markings based on the current position map-matched by the map information acquisition unit 12b. Also, when the driver inputs the destination, the target travel route is set along the travel route connecting the current location and the destination. This target traveling route is set from several hundred meters to several kilometers in front of the host vehicle M, and is updated as the vehicle travels. The vehicle control unit 26 reads the target traveling route set by the target traveling route setting calculation section 12c.

On the other hand, the stereo camera device 21 includes a camera section 22 as imaging means for imaging the front of the vehicle M, an image processing unit (IPU) 23, a camera image selection section 24 as camera image selection means, and a driving environment recognition means. is provided with a forward traveling environment recognition unit 25 of The camera unit 22 includes first to fourth cameras 22a to 22d of the same specifications, and as shown in FIG. position. Here, the second camera 22b and the third camera 22c constitute a short distance stereo camera, and the first camera 22a and the fourth camera 22d constitute a long distance stereo camera.

The second camera 22b and the third camera 22c are arranged at symmetrical positions with respect to the center in the vehicle width direction, and the short distance baseline length Ls is a relatively short distance stereo image (imaging area V in FIG. 5). is set to a value that can be obtained. Note that the stereo image in the present embodiment refers to an area from which a distance image can be obtained in the left and right images captured by a pair of cameras.

On the other hand, the first camera 22a and the fourth camera 22d are arranged at symmetrical positions on the driver's seat side and the passenger's seat side with the center in the width direction of the vehicle interposed therebetween. It is set to the long-distance baseline length Ll that can acquire the imaging region I). Images picked up by the respective cameras 22 a to 22 d are subjected to predetermined image processing by the IPU 23 and output to the camera image selection section 24 . The stereo image acquired for measuring the distance is not limited to the combination of the short-range stereo cameras 22b, 22c and the long-range stereo cameras 22a, 22d. can be acquired arbitrarily by selecting the image of the camera.

The camera image selection unit 24 selects the images captured by the cameras 22a to 22d transmitted from the IPU 23, based on the driving state and driving environment of the own vehicle M, and the front recognition that requires distance measurement with stereo images. Identify a pair of cameras that correspond to the region. Then, the image (stereo image) captured by the pair of cameras is transmitted to the forward running environment recognition section 25 .

Based on the stereo images (images captured by a pair of cameras) transmitted from the camera image selection unit 24, the forward traveling environment recognition unit 25 recognizes the left and right partition lines and the traveling route of the own vehicle M (own vehicle traveling route). Road shape, presence or absence of a preceding vehicle traveling in front of own vehicle M, three-dimensional objects including moving objects such as pedestrians and two-wheeled vehicles (bicycles, motorcycles) trying to cross in front of own vehicle M, signal appearance (lighting) color), road signs, etc. Then, from the focal length of the cameras, the baseline length between the cameras, and the parallax of the same object, the distance to the object is calculated using the principle of triangulation. Since the recognition of the object based on the stereo image and the method of obtaining the distance to the object are already known techniques, detailed description thereof will be omitted here.

The input side of the automatic driving control unit 26 is connected to the target travel route setting calculation section 12c of the map locator calculation section 12 and the forward traveling environment recognition section 25 of the stereo camera device 21 . Further, on the output side of the automatic driving control unit 26, there are a steering control unit 31 for driving the own vehicle M along the target travel route, a brake control unit 32 for decelerating the own vehicle M by forced braking, and a vehicle speed control unit 32 for controlling the speed of the own vehicle M. An acceleration/deceleration control unit 33 for control and an alarm device 34 are connected.

The automatic driving control unit 26 controls the steering control unit 31, the brake control unit 32, and the acceleration/deceleration control unit 33 in a predetermined manner, and controls the own vehicle M based on the positioning signal indicating the own vehicle position received by the GNSS receiver 13. The vehicle is automatically driven along the target travel route on the road map set by the target travel route setting calculation unit 12c. At that time, based on the forward running environment recognized by the forward running environment recognition unit 25, well-known following vehicle distance control (ACC: Adaptive Cruise Control) and lane keeping control (ALK: Active Lane Keep) are performed, and the preceding vehicle is detected. If detected, the vehicle will follow the preceding vehicle, and if the preceding vehicle is not detected, the vehicle will run within the speed limit. Furthermore, when a moving object that is about to cross in front of the own vehicle M is detected, the brake control unit 32 is operated to stop the own vehicle M.

By the way, as shown in FIG. 6, by combining the first to fourth cameras 22a to 22d of the camera section 22, stereo images (distance images) of the first to sixth imaging regions can be obtained. FIG. 13 shows the relationship between the selected pair of cameras and the front imaging area obtained from the stereo image.

If the baseline length is shortened, the distance to a short-distance object can be obtained with high accuracy. If the baseline length is lengthened, the distance to a distant object can be obtained with high accuracy.

Therefore, as shown in FIG. 7, the first imaging area I by the first camera 22a and the second camera 22d having the longest long-distance base line length Ll is an image of a long-distance three-dimensional object during high-speed driving on an expressway. It is suitable for detection and detection of lane markings on general roads with straight roads.

Further, as shown in FIG. 8, the second imaging area II formed by the first camera 22a and the third camera 22c can accurately detect an object at a medium distance slightly deviated to the left side of the own vehicle M. Therefore, it is suitable for detecting lane markings while driving on a left curve. On the other hand, as shown in FIG. 9, the third imaging area III by the second camera 22b and the fourth camera 22d can accurately detect an object at a medium distance slightly deviated to the right side of the own vehicle M. Therefore, it is suitable for detecting lane markings while driving on a right curve.

Further, as shown in FIG. 10, the fourth imaging area IV formed by the first camera 22a and the second camera 22b can accurately detect an object at a short distance slightly deviated to the left side of the own vehicle M. . Therefore, it is suitable for detecting moving bodies (pedestrians, two-wheeled vehicles, etc.) that try to cross in front of the own vehicle M from the left when traveling at low speed on a general road or starting from a stopped state.

Further, as shown in FIG. 11, the fifth imaging region V by the second camera 22b and the third camera 22c having the shortest short-distance base line length Ls is a short-distance three-dimensional object, lane markings, etc. during normal running. Suitable for detection. Further, as shown in FIG. 12, the sixth imaging area VI formed by the third camera 22c and the fourth camera 22d can accurately detect an object at a short distance slightly deviated to the right side of the host vehicle M. . Therefore, it is suitable for detecting moving objects (pedestrians, two-wheeled vehicles, etc.) that try to cross in front of the own vehicle M from the right direction when traveling at low speed on a general road or starting from a stopped state.

By the way, when the camera unit 22 is arranged close to the inside of the vehicle interior of the windshield as in the present embodiment, raindrops, snow, mud splashes, bird droppings, etc. adhere to the windshield and make it dirty. In this case, the field of view of each camera 22a-22d is obstructed. When trying to calculate the distance to an object from images captured by a pair of cameras, if the field of view of one camera is obstructed, the distance to the object cannot be calculated. Likewise, if one camera fails, the distance to the object cannot be determined.

Therefore, when the image of the camera is damaged due to the obstruction of the field of view or due to malfunction, the camera image selection unit 24 described above first identifies the defective camera, selects the corresponding substitute camera, The images captured by the normal camera and the substitute camera are output to the forward running environment recognition section 25 .

Specifically, camera image selection processing by the camera image selection unit 24 is executed according to a camera image selection processing routine shown in FIGS. In this routine, first, the travel information of the own vehicle M is acquired in steps S1 and S2. That is, in step S1, the running state information of own vehicle M is read. The traveling state information is the vehicle speed detected by the vehicle speed sensor constituting the autonomous traveling sensor 14, the yaw rate (yaw angular velocity) detected by the yaw rate sensor, and the longitudinal acceleration detected by the longitudinal acceleration sensor.

Further, in step S2, the surrounding environment information around the current location of the host vehicle M acquired by the map information acquiring section 12b of the map locator computing section 12 and around the front is read. This surrounding environment information includes static position information and dynamic position information as described above. Therefore, the autonomous travel sensor 14 and the map locator calculation unit 12 have the function of travel information acquisition means of the present invention.

Then, in step S3, based on the driving state information of the own vehicle M and the environment information of the road on which the own vehicle M is traveling and its surroundings, an imaging area for acquiring the required stereo image is selected. For example, when it is determined that the vehicle is traveling on a highway in the surrounding environment information, there is no traffic information, and the vehicle speed is high (e.g., 80 [Km/h] or more), the long-distance stereo camera 22a with the longest long-distance baseline length Ll , 22d (FIGS. 5, 7). On the other hand, even when traveling on a highway, the shortest is when the vehicle speed is medium to low speed (for example, about 10 to 40 [Km/h]), or when following the preceding vehicle P immediately before. A fifth imaging region (see FIGS. 5 and 11) by the short-range stereo cameras 22b and 22c having the short-range baseline length Ls is selected. Since other preferred examples have already been described, they are omitted.

In this way, by selecting two cameras from among the first to fourth cameras 22a to 22d according to the driving state and the driving environment, the first to sixth imaging regions are shown in FIGS. Stereo images can be obtained at I-VI. In step S3 described above, the imaging region may be selected based on one of the driving state information of the own vehicle M and the surrounding environment information.

After that, the process proceeds to step S4 to obtain a stereo image captured by a pair of cameras corresponding to the selected imaging area, and the process proceeds to step S5 to perform stereo matching processing of the stereo image. Since this stereo matching processing is well known, the description thereof is omitted here.

After that, in step S6, it is checked whether or not there is a non-matching image in the image of the corresponding section in the left and right images in the stereo image matching process. It should be noted that the processing in steps S5 and S6 corresponds to the camera failure determination means of the present invention.

If no non-matching image is detected, it is determined to be normal, the process proceeds to step S7, the current stereo image is transmitted to the forward running environment recognition section 25, and the routine exits. On the other hand, if a non-matching image is detected in step S6, it is determined as a failure, and the process jumps to step S8.

Proceeding to step S8, the defective camera is identified. In the case of a stereo camera (a pair of cameras), if the camera itself is out of order, no image signal is sent to the camera image selection unit 24, so the malfunctioning camera can be easily identified. However, if the image is partially masked due to dirt on the windshield or camera lens, or if the image is blurred due to raindrops adhering to the windshield, the stereo images captured by both cameras will be compared. However, it is not possible to determine which camera is defective.

Therefore, in step S8, other combinations including the camera selected in step S3 are selected one by one, stereo matching is performed on each pair in the same manner as in step S5, and the defective camera is identified. do. That is, as shown in FIG. 13, when the first camera 22a fails, the stereo images of the first, second and fourth imaging areas I, II and IV cannot be obtained. If the second camera 22b fails, stereo images of the third to fifth imaging areas III to V cannot be acquired. If the third camera 22c fails, the stereo images of the second, fifth and sixth imaging areas II, V and VI cannot be obtained. Also, if the fourth camera 22d fails, the stereo images of the first, third and sixth imaging areas I, III and VI cannot be acquired.

Therefore, when a defect is detected in a stereo image by a pair of cameras, first, a stereo image is individually obtained from a combination of one of the pair of cameras and the other camera. Then, stereo matching processing is performed on each of the acquired stereo images, and by checking in which pair of cameras (stereo cameras) the inconsistent image is detected, the defective camera is specified. In this case, since an image at a short distance is clearer than an image at a long distance, a combination of imaging areas on the short distance side is selected as much as possible.

A specific combination is shown in FIG. That is, when the stereo image of the first imaging area I is defective, the stereo images of the fourth and sixth imaging areas IV and VI are acquired and stereo matching processing is performed, and the images of the first and fourth cameras 22a and 22d are obtained. Identify what went wrong. Further, when the stereo image of the second imaging area II is defective, the stereo images of the fourth and fifth imaging areas IV and V are acquired and stereo matching processing is performed, and the images of the first and third cameras 22a and 22c are obtained. Identify what went wrong. Furthermore, when the stereo image of the third imaging region III is defective, the stereo images of the fifth and sixth imaging regions V and VI are acquired and stereo matching processing is performed, and the images of the second and fourth cameras 22b and 22d are obtained. Identify what went wrong.

On the other hand, when the stereo image of the fourth imaging area IV is defective, the stereo images of the second and fifth imaging areas II and V are acquired and stereo matching processing is performed, and the images of the first and second cameras 22a and 22b are obtained. Identify what went wrong. Furthermore, when the stereo image of the fifth imaging region V is defective, the stereo images of the fourth and sixth imaging regions IV and VI are acquired and stereo matching processing is performed, and the images of the second and third cameras 22b and 22c Identify what went wrong. Further, when the stereo image of the sixth imaging area VI is defective, the stereo images of the third and fifth imaging areas III and V are acquired and stereo matching processing is performed, and the images of the third and fourth cameras 22c and 22d are obtained. Identify what went wrong.

Then, after identifying the failed camera in step S8, the process proceeds to step S9 to set a substitute imaging area and a substitute monocular camera by a camera to be applied in place of the failed camera. As shown in FIG. 14, when one of the pair of cameras used in the normal imaging area fails, the first to third imaging areas I to III on the long-distance side are closest to the imaging area. is selected as the alternative imaging region. On the other hand, if the camera that captures the fourth to sixth imaging areas IV to VI on the short distance side fails, the alternative imaging area will be replaced by a moving object that is about to cross the immediately preceding preceding vehicle or the own vehicle M. (Pedestrians, motorcycles, etc.) will not be able to recognize the distance.

Therefore, in the fourth to sixth imaging areas IV to VI, if one of the cameras fails, the process may be interrupted (HALT). is selected as an alternative monocular camera. Therefore, in this case, the image to be transmitted is a monocular image.

[First imaging area I]
If the first camera 22a fails, the third imaging area III closest to the first imaging area I and using the fourth camera 22d is selected as an alternative imaging area. Also, if the fourth camera 22d fails, the second imaging area II closest to the first imaging area I and using the first camera 22a is selected as an alternative imaging area.

[Second imaging area II]
If the first camera 22a fails, the fifth imaging area V closest to the second imaging area II and using the third camera 22c is selected as an alternative imaging area. Also, when the third camera 22c fails, the fourth imaging area IV closest to the second imaging area II and using the first camera 22a is selected as an alternative imaging area.

[Third imaging area III]
If the second camera 22b fails, the fourth imaging area IV closest to the third imaging area III and using the fourth camera 22d is selected as an alternative imaging area. Also, if the fourth camera 22d fails, the fifth imaging area V closest to the third imaging area III and using the second camera 22b is selected as an alternative imaging area.

[Fourth imaging area IV]
If the first camera 22a fails, the normal second camera 22b is selected as a substitute monocular camera. Conversely, if the second camera 22b fails, the first camera 22a is selected as an alternative monocular camera.

[Fifth imaging area V]
If the second camera 22b fails, the normal third camera 22c is selected as a substitute monocular camera. Conversely, if the third camera 22c fails, the second camera 22b is selected as an alternative monocular camera.

[Sixth imaging area VI]
If the third camera 22c fails, the normal fourth camera 22d is selected as an alternative monocular camera. Conversely, if the fourth camera 22d fails, the third camera 22c is selected as an alternative monocular camera.

Next, the process proceeds to step S10 to check whether the selected alternative imaging area or an alternative monocular camera is selected. If the alternative imaging area is selected, the process advances to step S11. Also, in the case of the alternative monocular camera, the process jumps to step S12.

When proceeding to step S11, a stereo image captured by a pair of cameras corresponding to the selected alternative imaging area is acquired, and the process proceeds to step S13. Further, when proceeding to step S12, a monocular image captured by the selected alternative monocular camera is acquired, and the process proceeds to step S13. It should be noted that the processing in steps S8 to S12 corresponds to the substitute image obtaining means of the present invention.

After that, when proceeding from step S11 or step S12 to step S13, the current stereo image or monocular image is sent to the forward running environment recognition unit 25 in place of the defective stereo image, and the routine is exited. If the malfunction of the camera is temporary, when the malfunction is recovered, the process proceeds from step S6 to step S7, and the stereo image acquired in step S4 is transmitted.

The forward traveling environment recognition unit 25 reads the stereo image or the monocular image transmitted from the camera image selection unit 24 . Then, the same object in both images is recognized based on the parallax from the stereo image, and the distance data (the distance from the own vehicle M to the object) is calculated using the principle of triangulation. is recognized as forward running environment information. On the other hand, in addition to recognizing the object from the monocular image, the distance to the object is calculated from the change in size between the object recognized this time and the same object recognized in the previous calculation, and these are then moved forward. Recognize as environmental information.

Based on the forward running environment information recognized by the forward running environment recognition unit 25, the automatic driving control unit 26 transmits a control signal to the steering control unit 31 so that the host vehicle M runs along the center of the left and right lane markings. Further, when the preceding vehicle P traveling immediately ahead is recognized, a control signal is transmitted to the brake control section 32 and the acceleration/deceleration control section 33 so as to allow the preceding vehicle P to travel with a predetermined inter-vehicle distance.

In addition, when camera failure is detected by the camera image selection unit 24 of the stereo camera device 21, the warning device 34 notifies the driver to that effect, and when it becomes difficult to continue automatic driving support, Prepare the driver to take over driving. In addition, when the failure is recovered, the driver is notified of this fact.

As described above, in the present embodiment, when one of the four cameras 22a to 22d fails, an object based on a stereo image of a specific imaging area captured by a pair of cameras including the camera is detected. Even if the distance information to the vehicle cannot be continuously maintained, the closest imaging area including a normal camera is set as an alternative imaging area, and the forward driving environment is based on the stereo image captured in this alternative imaging area. Therefore, even if the camera that has failed is replaced with another camera, the driving state is not significantly impaired, and driving support control and automatic driving support can be continued. As a result, even if it is determined that automatic driving assistance is difficult, the driver can smoothly take over the driving, and the sense of discomfort given to the driver can be reduced.

Further, which one of the pair of cameras is defective is specified based on the stereo image obtained by capturing the other imaging area including one of the pair of cameras, so that the malfunction occurs during running. A camera can be identified. Therefore, it is not necessary to stop the own vehicle M to check the failed camera, and high convenience can be obtained. Furthermore, since the automatic driving support can be continued when the failure is recovered during driving, higher convenience can be obtained.

The present invention is not limited to the above-described embodiment. For example, the camera section 22 may be composed of five or more cameras, and the imaging area is set by combining a pair of cameras among them. You can make it work. Further, the camera unit 22 may take images of not only the front of the own vehicle M but also the outside traveling environment including the rear. Therefore, the camera unit 22 may be arranged on the indoor side of the rear glass of the vehicle M to capture the rear running environment. do.

1... automatic driving support system,
11... Locator unit,
12 ... map locator calculation unit,
12a ... own vehicle position estimation calculation unit,
12b ... map information acquisition unit,
12c ... target course setting calculation unit,
13 ... GNSS receiver,
14... Autonomous traveling sensor,
16... high-precision road map database,
21... Stereo camera device,
22... camera section,
22a to 22d... first to fourth cameras,
24... Camera image selection unit,
25 ... forward running environment recognition unit,
26 ... automatic operation control unit,
26... vehicle control unit,
26 ... automatic operation control unit,
31 ... steering control section,
32 ... brake control section,
33 Acceleration/deceleration control unit,
34 ... alarm device,
Ll: long-distance baseline length,
Ls: short-range baseline length,
M... own vehicle,
P... Leading vehicle,
I to VI: 1st to 6th imaging areas

Claims (6)

a travel information acquiring means for acquiring travel information of the own vehicle;
an imaging means having at least a pair of long-distance cameras and a pair of short-distance cameras arranged on the own vehicle at a predetermined base line length to pick up images of the outside of the own vehicle;
One of the pair of long-distance cameras and the pair of short-distance cameras of the imaging means is selected based on the travel information acquired by the travel information acquisition means, and an image is captured by the selected pair of cameras. camera image selection means for acquiring stereo images of a region;
Driving environment recognition for recognizing the driving environment outside the own vehicle based on a stereo image from one of the pair of long-distance cameras and the pair of short-distance cameras selected by the camera image selection means. In a vehicle stereo camera device having means,
The camera image selection means is
camera failure determination means for performing stereo matching processing on the stereo images acquired from the selected pair of cameras to determine whether or not the images are defective;
When the camera defect determination means determines that the image is defective, it is checked whether the pair of cameras for long distance or the pair of cameras for short distance has been selected, and the pair of cameras for long distance is checked. When it is determined that one of the cameras is defective, a substitute camera for the malfunctioning camera is selected from cameras other than the selected pair of cameras, and the normal state of the substitute camera and the pair of cameras is determined. When it is determined that one of the pair of short-distance cameras is defective, the image is taken by the normal camera of the pair of short-distance cameras. A stereo camera device for a vehicle, comprising a substitute image acquiring means for acquiring an image as a monocular image . 2. The vehicle stereo camera apparatus according to claim 1, wherein the running information acquired by the running information acquiring means is running state information of the own vehicle. 2. The vehicle stereo camera apparatus according to claim 1, wherein the traveling information acquired by the traveling information acquiring means is environment information around the own vehicle. 2. The vehicle stereo camera device according to claim 1, wherein the running information acquired by said running information acquiring means is both the running state information of said own vehicle and the environment information around said own vehicle. When the camera failure determination means determines that the image is defective, the substitute image acquisition means newly selects another pair of combinations separately including the selected pair of cameras. 4. The vehicle stereo according to any one of claims 1 to 4, wherein the stereo images obtained from each pair of cameras are subjected to stereo matching processing to identify the camera with the image defect. camera equipment. 6. The vehicle stereo camera apparatus according to claim 1, wherein the alternative image acquisition means sets an imaging area closest to the imaging area as the alternative imaging area.

Images