JP4799236B2 - In-vehicle display system - Google Patents

Description

  The present invention relates to an in-vehicle display system that displays information to be presented to a driver.

For example, Patent Document 1 discloses a pedestrian protection device that suppresses a secondary collision to an engine or the like by lifting the rear of the hood upward when the pedestrian first collides with a front bumper or the like. . Further, for example, Patent Document 2 discloses a system that displays an image in front of a vehicle captured by a monocular camera on a display when an airbag on a hood is deployed to protect a pedestrian who has collided primarily. Yes. This monocular camera is installed in a rearview mirror, and a notch is formed in a part of the airbag so that the camera view is not blocked by the airbag in the deployed state.
Japanese Patent No. 3340704 Japanese Patent Laid-Open No. 2004-284440

  However, in the prior art of Patent Document 2 described above, a case in which almost the entire camera field of view is blocked by an obstacle such as a pedestrian can easily occur due to the use of a monocular camera. Under such circumstances, it is difficult to effectively obtain an image ahead of the vehicle, and it is difficult to present information that can compensate for the poor visibility of the driver to the driver. This problem can be solved to some extent, for example, by installing a monocular camera in the grill, but on the other hand, a new problem arises that the camera is easily damaged by an impact at the time of collision.

  The present invention has been made in view of such circumstances, and an object thereof is to more reliably acquire and present information in front of the vehicle when the driver's field of view is blocked by a pedestrian or the like who has collided primarily.

  Another object of the present invention is to provide a stereo camera with multiple functions and high added value by adding a new use for handling a collision to the stereo camera.

  In order to solve this problem, the present invention provides an in-vehicle display system including a stereo camera, a display device, a display determination unit, and a display control unit. The stereo camera captures a scene in front of the vehicle and outputs a pair of captured images. The display device displays information to be presented to the driver. When the vehicle collides with an object ahead, the display determination unit changes the luminance in the frame between the pair of captured images output from the stereo camera, or the time-series luminance between the frames for each of the pair of captured images. Based on the change, a camera whose camera view is not obstructed is identified. The display control unit causes the display device to display a captured image of the camera whose field of view is not obstructed according to the determination result of the display determination unit.

  Here, in the present invention, when the vehicle collides with an object in front of the vehicle, in order to buffer the impact acting on the object, the operating state that blocks the driver's field of view from the non-operating state that does not block the driver's field of view toward the front of the vehicle You may provide further the object protection part from which a form changes. In this case, it is preferable that the stereo camera is attached at a position where the camera view is not obstructed by the object protection unit that is in the activated state.

In the present invention, information in front of the vehicle acquired by a device other than the stereo camera may be input to the display determination unit. In this case, the display determination unit selects either the information ahead of the vehicle from the stereo camera or the information ahead of the vehicle from a device other than the stereo camera according to the display priority order, and displays the selected information on the display device. It is preferable to instruct the display control unit to do so. The information in front of the vehicle preferably includes at least one of radar information output from the vehicle-mounted radar device and reception information acquired from outside the vehicle by vehicle-to-vehicle communication or road-to-vehicle communication. The display determination unit may select radar information as information to be displayed on the display device when a camera whose camera view is not blocked cannot be specified . In addition, when receiving the reception information, the display determination unit may select the reception information as information to be displayed on the display device regardless of whether or not the camera view of the stereo camera is blocked.

  In the present invention, it is preferable that the display determination unit instructs the display control unit to continue image display on the display device at least until the vehicle stops.

  According to the present invention, a camera in which the camera view is not obstructed by a pedestrian or the like is specified by monitoring a time-series luminance change between frames regarding each captured image from the stereo camera. This is based on the knowledge that, for example, when the airbag is activated or a pedestrian collides with the vehicle, the captured image changes rapidly between frames. Thus, even in the event of a collision, the situation ahead of the vehicle can be effectively presented to the driver via the display device, and the driver's visibility defect can be more reliably compensated. In addition, by adding a new application such as displaying the situation in front of the vehicle at the time of a collision to the stereo camera, it becomes possible to increase the functionality and added value of the stereo camera.

  FIG. 1 is a schematic configuration diagram of an in-vehicle display system. In particular, this system includes a pedestrian protection device including a collision sensor 1, a collision determination unit 2, an operation control unit 3, and an object protection unit 4, a stereo camera 5, a control unit 6, and a display device 7. . The pedestrian protection device inflates a cushioning member such as an air bag in order to alleviate a secondary collision of a pedestrian who has primarily collided with a front bumper or the like. The secondary collision refers to a phenomenon in which a pedestrian who has collided first collides with a highly rigid engine or the like via a hood. The collision sensor 1 is a pressure-sensitive type or contact switch type sensor provided in a bumper or the like, and detects a collision with an obstacle in front of the vehicle. More specifically, the backward acceleration is detected from the external force applied to the vehicle at the time of the collision, and this detected value is output to the collision determination unit 2. The collision determination unit 2 calculates the deformation speed of the bumper based on the detected acceleration, and determines whether the collision object is a pedestrian based on the calculated value or the like. This determination result is output to the operation control unit 3 and the control unit 6. In addition, since the detailed determination method of a pedestrian collision is explained in detail in, for example, Japanese Patent No. 3340704, refer to it if necessary.

  The operation control unit 3 is configured by a known inflator or the like, and operates the object protection unit 4 based on the determination result from the collision determination unit 2. As the object protection unit 4, a hood airbag or an active hood can be typically used. When the former is used, it is stored in the hood in its non-operating state (normal time), and when it is in the operating state (at the time of pedestrian collision), it instantly expands and deploys on the hood. The hood airbag does not block the driver's view in the stored non-operating state, but blocks the driver's view to the front of the vehicle in the deployed operating state. Note that the point of view of the driver being blocked when the form changes to the operating state is the same even when an active hood is used as the object protection unit 4.

  The stereo camera 5 includes a pair of cameras, and outputs a pair of captured images by capturing a scene in front of the vehicle. Each camera incorporates an image sensor such as a CCD or a CMOS, and the light received by the image sensor is converted into an electrical signal and output. The stereo camera 5 is attached to a position where the camera field of view is not obstructed by the object protection unit 4 in the activated state, for example, in the vicinity of the room mirror.

  The control unit 6 performs forward monitoring processing with the stereo camera 5 as an input in normal times, and displays a captured image acquired by the stereo camera 5 on the display device 7 when it collides with an object such as a pedestrian.

  FIG. 2 is a block diagram of the control unit 6. The control unit 6 is based on a normal processing system composed of blocks 6a to 6d, and has a configuration in which a collision processing system composed of blocks 6e and 6f is added. As the input system of the control unit 6, the stereo camera 5 and the collision determination unit 2 are necessary, but a radar device 8 and a communication device 9 may be connected in addition to this. As the radar device 8, a millimeter wave radar which is one of radio wave radars can be used, and is usually attached at a position different from the stereo camera 5, for example, inside the front bumper. The communication device 9 receives information from the outside of the vehicle (vehicles other than the host vehicle or road infrastructure) by inter-vehicle communication or road-to-vehicle communication. Information in front of the vehicle from the radar device 8 and the communication device 9 is input to the control unit 6.

  With respect to the normal processing system, a pair of captured images from the stereo camera 5 is stored in the image data memory 6a in units of frames and is output to the stereo processing unit 6b. The stereo processing unit 6b performs stereo matching processing using a pair of captured images as inputs, and calculates distance data that is a set of parallaxes associated with positions on the image plane. The calculated distance data is stored in the distance data memory 6c in units of frames. Based on the data read from the distance data memory 3c and, if necessary, the data read from the image data memory 6a, the object recognition unit 6d detects the presence of an object such as a preceding vehicle or a pedestrian existing in front of the vehicle, and , Recognize the position in real space. And according to this recognition result, various vehicle controls, such as braking to a brake and braking by downshifting, are performed, for example, in order to secure the distance between the preceding vehicle.

  On the other hand, the collision processing system including the blocks 6e and 6f is for the driver whose field of view is blocked by the object protection unit 4 when the collision determination unit 2 detects a collision with an object (typically a pedestrian). Thus, information ahead of the vehicle is presented via the display device 7. The display determination unit 6e selects an image to be presented to the driver based on the captured image of the current frame output from the stereo camera 5 and the captured image of the past frame read from the image data memory 6a. In order to display this on the display device 7, the selection control unit 6f is notified of the selection result.

  Hereinafter, as a collision display method, two cases, that is, a case using only the stereo camera 5 (first embodiment) and a case using a device other than the stereo camera 5 together (second embodiment) are given. Each will be described in detail.

(First embodiment)
FIG. 3 is a flowchart of the display processing at the time of collision according to the first embodiment. This process is repeatedly executed at predetermined intervals. First, in step 1, it is determined whether or not a collision has occurred. That is, when the determination result of the collision determination unit 2 is “collision”, the display determination unit 6e shifts the process to the subsequent step 3, while when the determination result is “no collision”, the display determination unit 6e The process is skipped and the process in the current cycle is terminated.

  In step 3, the display determination unit 6e determines a time-series luminance change ΔBr between frames related to the right image output from the right camera in the stereo camera 5, and this luminance change ΔBl related to the left image output from the left camera. Is calculated. These luminance changes ΔBr and ΔBl may be an average luminance change (absolute value) of the entire screen (or a region similar thereto), or may be a sum of luminance differences (absolute values) between frames of each pixel. Good. When the air bag is activated or the pedestrian collides with the vehicle, the captured image changes abruptly between frames. Therefore, if the luminance changes ΔBr and ΔBl reflecting such a time-series change, The specific calculation method is not limited to this.

  In step 4, the display control unit 6e determines whether or not the luminance change ΔBr of the right image is smaller than a predetermined determination threshold value ΔBth. A small time-series luminance change of the right image means that the degree of the reflection of the shielding object (pedestrian etc.) in the right image is small, in other words, the right camera is not shielded so much. . Therefore, when an affirmative determination is made in step 4, the right image with a small degree of reflection of the shielding object is selected, and the display control unit 6f is instructed to display the right image (step 6). Upon receiving this instruction, the display control unit 6f interrupts the display of the normal image (for example, navigation display), and causes the display device 7 to display the image in front of the vehicle captured by the right camera.

  On the other hand, when a negative determination is made in step 4, the display control unit 6e determines whether or not the luminance change ΔBl of the left image is smaller than the determination threshold value ΔBth (step 5)). The small time-series luminance change of the left image means that the degree of the shielding object in the left image is small. Therefore, in this case, the left image picked up by the left camera whose camera view is not well shielded is selected, and the display control unit 6f is instructed to display the left image (step 7). Upon receiving this instruction, the display control unit 6f causes the display device 7 to display an image in front of the vehicle captured by the left camera, instead of displaying a normal image.

  On the other hand, if a negative determination is made in step 5, the right and left images are not necessarily optimal as information for informing the driver of the situation ahead of the vehicle, but the right image display is instructed as a good workaround (step 6). .

  Note that the image display ahead of the vehicle resulting from steps 6 and 7 is preferably continued at least until the vehicle stops. And when a vehicle stops, it returns to the display of a normal screen.

  As described above, in this embodiment, the camera view is blocked based on the time-series luminance change between the frames relating to each of the pair of captured images output from the stereo camera 5 at the time of collision with an object such as a pedestrian. A camera that is not connected is determined. And according to this determination result, the captured image of the camera whose camera view is not obstructed is displayed on the display device 7. Thereby, even when the developed object protection unit 4 or the pedestrian becomes a shielding object and the visibility of the driver is poor, the situation in front of the vehicle can be accurately presented to the driver via the display device 7. In particular, in the stereo camera 5, the possibility that any camera field of view is blocked is lower than when a monocular camera is used. Therefore, in this embodiment, a driver's visual field defect can be compensated more reliably in comparison with a monocular camera.

  In addition, according to the present embodiment, the stereo camera 5 is added with not only the original use of controlling the inter-vehicle distance with the preceding vehicle but also the new use of displaying the situation in front of the vehicle at the time of a collision. The camera 5 can be further multifunctional and have high added value.

  In the present embodiment, a determination method is used based on a time-series luminance change between frames relating to each of a pair of captured images, but the present invention is not limited to this. That is, the determination based on the luminance change between images is the essence of the invention, and the determination may be based not only on the time-series luminance change but also on the luminance change in the frame between a pair of captured images.

  For example, when only one edge to be detected in both images is detected based on the edge (brightness change point) detected in each of the left and right images (frame images), the detected side is not shielded. Judge that it is. This is a determination using the fact that an edge cannot be extracted when the field of view is blocked. The determination of the edge is preferably performed based on the presence or absence of, for example, a travel line that is detected as a luminance change point on the road surface. That is, when a white line is detected in one image but no white line is detected in the other image, it is determined that the image on the side detecting the white line is not occluded. Use this. As a specific edge detection method, for example, when a luminance difference between adjacent pixels in one direction (for example, the horizontal direction) is larger than a predetermined threshold value, the portion can be regarded as an edge.

  In addition, for example, the overall luminance magnitude of a partial monitoring area set to substantially the whole or a part of one image (frame image) (preferably a relatively large area) is set to the other image. If the difference in size between the two differs by a predetermined threshold value or more, the image with higher luminance may be regarded as an unshielded image and used.

(Second Embodiment)
FIG. 4 is a flowchart of the collision time display process according to the second embodiment. The difference from the flowchart of FIG. 3 described in the first embodiment is that steps 2, 8, and 9 are added. Since the other points are the same as those in FIG. 2, the same step numbers are assigned and the description thereof is omitted here.

  In the case of “collision” in step 1, the display determination unit 6e determines whether information is received from outside the vehicle via the communication device 9 (step 2). This reception information is acquired from outside the vehicle by vehicle-to-vehicle communication or road-to-vehicle communication, and is information indicating the situation ahead of the vehicle. If an affirmative determination is made in step 2, the reception information of the communication device 9 is selected, and the display control unit 6f is instructed to display the reception information (step 8). Upon receiving this instruction, the display control unit 6f causes the display device 7 to display the received information instead of displaying the normal image. That is, when there is reception information indicating the situation ahead of the vehicle, the reception information of the communication device 9 is selected as a display target regardless of whether or not the camera view of the stereo camera 5 is blocked. On the other hand, if a negative determination is made in step 2, the process proceeds to step 3 and subsequent steps in order to display the captured image of the stereo camera 5.

  When a negative determination is made in step 5, that is, when neither of the left and right images is optimal as information for informing the driver of the situation ahead of the vehicle, display of radar information acquired by the radar device 8 is instructed ( Step 9). Upon receiving this instruction, the display control unit 6f causes the display device 7 to display radar information instead of displaying the normal image. In general, when laser information is displayed as an image, an object in front of the vehicle can be detected although the sharpness is lower than that of a camera image. When it is determined that the camera view of the stereo camera 5 is blocked, radar information is selected as information to be displayed on the display device 7. Even in the display of radar information, the purpose of presenting information to the driver is achieved to some extent.

  As described above, in the present embodiment, any information is selected according to the display priorities such as (1) the reception information of the communication device 9, (2) the captured image of the stereo camera, and (3) the radar information of the radar device 8. Is displayed on the display device 7. As a result, even when a driver's visibility failure occurs at the time of a collision, the possibility that information indicating the situation ahead of the vehicle can be appropriately displayed increases, so that the driver's visibility failure can be more reliably compensated.

  In the above-described embodiment, a case has been described in which both radar information from the radar device 8 and reception information from the communication device 9 are used as the information ahead of the vehicle, but only one of them may be used. Well, the display priority is not limited to the above case.

Schematic configuration diagram of in-vehicle display system Block diagram of control unit Flowchart of collision display processing according to the first embodiment Flowchart of collision display processing according to the second embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Collision sensor 2 Collision determination part 3 Operation control part 4 Object protection part 5 Stereo camera 6 Control unit 7 Display apparatus 8 Radar apparatus 9 Communication apparatus 3a Image data memory 3b Stereo processing part 3c Distance data memory 3d Object recognition part 3e Display determination part 3f Display control unit

Claims (7)

In in-vehicle display system,
A stereo camera that images the scenery in front of the vehicle and outputs a pair of captured images;
A display device for displaying information to be presented to the driver;
When a vehicle collides with an object ahead, based on a change in luminance within a frame between a pair of captured images output from the stereo camera or a time-series luminance change between frames for each of a pair of captured images A display determination unit for identifying a camera whose camera view is not obstructed,
An in-vehicle display system comprising: a display control unit that causes the display device to display a captured image of a camera whose field of view is not obstructed according to a determination result of the display determination unit. An object whose form changes from a non-operating state that does not block the driver's field of view to the front of the vehicle to an operating state that blocks the driver's field of view in order to buffer the impact acting on the object when the vehicle collides with an object in front It further has a protection part,
The in-vehicle display system according to claim 1, wherein the stereo camera is attached at a position where a camera view is not obstructed by the object protection unit in the activated state. Information on the front of the vehicle acquired by a device other than the stereo camera is also input to the display determination unit,
The display determination unit selects either the information in front of the vehicle from the stereo camera or the information in front of the vehicle from a device other than the stereo camera according to the priority of display, and the selected information is displayed on the display device. The in-vehicle display system according to claim 1 or 2, wherein the display control unit is instructed to display.   The information ahead of the vehicle includes at least one of radar information output from a vehicle-mounted radar device and reception information acquired from outside the vehicle by vehicle-to-vehicle communication or road-to-vehicle communication. In-vehicle display system. The in-vehicle display system according to claim 4, wherein the display determination unit selects the radar information as information to be displayed on the display device when a camera whose camera view is not obstructed cannot be specified .   The display determination unit, when receiving the reception information, selects the reception information as information to be displayed on the display device regardless of whether the camera view of the stereo camera is blocked or not. The in-vehicle display system according to claim 4 or 5.   The in-vehicle display according to any one of claims 1 to 6, wherein the display determination unit instructs the display control unit to continue image display on the display device at least until the vehicle stops. system.

Images