JP2011188028A - Vehicle surrounding monitoring system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle surrounding monitoring system that improves user-friendliness for a user. <P>SOLUTION: The vehicle surrounding monitoring system includes a relative relationship change detection unit 82 which detects change of relative relationship between a display surface of a display device 5 and a driver viewpoint relevant position which is a position of a part relevant to a viewpoint of a driver of a self-vehicle, wherein a display control unit 85 which an image like a rearview mirror which is the image equivalent to an image to be reflected on the rearview mirror of the vehicle is displayed on the display device 5 shifts a range of the image like the rearview mirror to be displayed on the display device 5 according to change of the relative relationship detected by the relative relationship change detection unit 82. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a vehicle periphery monitoring system.

  Conventionally, the rear and rear side areas of the vehicle projected by a rear-view mirror consisting of optical mirrors such as door mirrors, fender mirrors, and room mirrors are imaged with a camera, and the captured images are displayed on a monitor installed in the vehicle interior. Electronic mirror systems for display are being studied.

  For example, in Patent Document 1, a camera having an imaging range wider than the field of view of the door mirror is installed on the front side of the door mirror, and the viewpoint conversion direction is determined from the viewpoint of the driver obtained from the position of the driver's seat and the reflection angle of the door mirror. An electronic mirror system is disclosed in which a captured image of a camera is converted into an image at the same viewpoint as an image projected on a door mirror and displayed on a display unit.

Japanese Patent No. 3877761

  With optical mirrors, when the driver wants to change the field of view behind or behind, the mirror angle is adjusted manually or by motor drive, or the driver moves the viewpoint by moving the body or head, The field of view was changed.

  However, in the electronic mirror system disclosed in Patent Document 1, the range of the captured image displayed on the display unit is determined from the position of the driver's seat and the reflection angle of the door mirror. Even if the viewpoint is moved by moving the body or head, the range of the captured image displayed on the display unit does not change. As described above, the electronic mirror system disclosed in Patent Document 1 has a problem that the usability for the driver is lower than that of the optical mirror.

  The present invention has been made in view of the above-described conventional problems, and an object of the present invention is to provide a vehicle periphery monitoring system that can further improve usability for a driver.

  In a rear-view mirror consisting of an optical mirror, the range of the image reflected on the reflection surface of the rear-view mirror is a change in the relative relationship between the reflection surface of the rear-view mirror and parts related to the viewpoint such as the eyes and face of the driver of the vehicle. It shifts according to.

  According to the vehicle periphery monitoring system of claim 1, the change in the relative relationship between the display surface of the display unit detected by the relative relationship change detection unit and the driver viewpoint related position which is the position of the portion related to the viewpoint of the driver of the vehicle. Since the range of the rearview mirror-like image (image corresponding to the image displayed on the rearview mirror of the vehicle) to be displayed on the display unit is shifted according to the driver, when the driver moves the body or head, The range of the rearview mirror-like image displayed on the display unit can be shifted in the same manner as the shift of the range of the image reflected on the reflection surface of the rearview mirror.

  Also, when the driver moves the body or head to move the viewpoint, the range of the rearview mirror-like image displayed on the display unit is shifted in the same way as the shift of the range of the image reflected on the reflection surface of the rearview mirror. As a result, the same usability as that of the rearview mirror can be realized in the vehicle periphery monitoring system. Therefore, according to the above configuration, the usability of the vehicle periphery monitoring system for the driver can be further improved. Become.

  According to another aspect of the present invention, a driver imaging unit that images a driver of a vehicle and a driver viewpoint-related position detection unit that detects a driver viewpoint-related position from a driver image captured by the driver imaging unit are provided. Based on the driver viewpoint related position detected by the driver viewpoint related position detection unit, as a change in the relative relationship between the display surface of the display unit and the driver viewpoint related position, It is good also as an aspect which detects the change of a related position.

  In the rearview mirror composed of an optical mirror, the range of the image reflected on the reflection surface of the rearview mirror moves up, down, left and right with respect to the reflection surface of the rearview mirror. In this case, the direction related to the viewpoint of the driver is shifted in the direction opposite to the moving direction.

  According to the configuration of claim 3, the display unit displays the change in the driver viewpoint related position with respect to the display surface of the display unit detected by the relative relationship change detection unit in any of the vertical and horizontal directions. Since the range of the rear-view mirror-like image is shifted in the direction opposite to the direction of the change, the range of the rear-view mirror-like image to be displayed on the display unit is similar to the shift of the range of the image shown on the reflection surface of the rear-view mirror. Can be shifted. Therefore, according to the above configuration, the usability of the vehicle periphery monitoring system for the driver can be further improved.

  Further, according to the configuration of claim 4, by the amount corresponding to the amount of change in the vertical or horizontal direction of the driver viewpoint related position with respect to the display surface of the display unit detected by the relative relationship change detection unit, Since the range of the rearview mirror-like image displayed on the display unit is shifted in the direction opposite to the direction of the change, the range of the rearview mirror-like image displayed on the display unit is the range of the image reflected on the reflection surface of the rearview mirror. It can be shifted in a manner more similar to the shift. Therefore, according to the above configuration, the usability of the vehicle periphery monitoring system for the driver can be further improved.

  Further, as in claim 5, the display surface of the display unit can be changed in direction by at least one of manual operation and electric operation, and the display surface direction detection unit detects the direction of the display surface. The relative relationship change detection unit detects the change in the relative relationship between the display surface of the display unit and the driver viewpoint related position based on the orientation of the display surface detected by the display surface direction detection unit. On the other hand, it is good also as an aspect which detects the change of direction of the display surface of the display part.

  Note that in the rearview mirror consisting of an optical mirror, the range of the image reflected on the reflection surface of the rearview mirror is such that the direction of the reflection surface of the rearview mirror is up, down, left, and right with respect to the portion related to the viewpoint of the vehicle driver. When it moves, it shifts in the same direction as the reflecting surface of the rearview mirror.

  According to the configuration of the sixth aspect, the display unit responds to a change in the direction of the display surface of the display unit relative to the driver viewpoint related position detected by the relative relationship change detection unit in any of the vertical and horizontal directions. Since the range of the rearview mirror-like image to be displayed is shifted in the same direction as the direction of the change, the rearview mirror-like image to be displayed on the display unit is displayed in the same manner as the deviation of the range of the image reflected on the reflection surface of the rearview mirror. The range can be shifted. Therefore, according to the above configuration, the usability of the vehicle periphery monitoring system for the driver can be further improved.

  According to the configuration of claim 7, the amount corresponding to the amount of change in the direction of the display surface of the display unit relative to the driver viewpoint related position detected by the relative relationship change detection unit in any of the upper, lower, left and right directions. However, since the range of the rearview mirror-like image displayed on the display unit is shifted in the same direction as the direction of the change, the range of the rearview mirror-like image displayed on the display unit is changed to the image reflected on the reflection surface of the rearview mirror. It can be shifted in a manner more similar to the shift in range. Therefore, according to the above configuration, the usability of the vehicle periphery monitoring system for the driver can be further improved.

  Further, as in an eighth aspect, the driver viewpoint related position may be any one of the face and eyes of the driver of the vehicle.

  According to another aspect of the present invention, when the display control unit detects an obstacle with the first obstacle detection unit, the display control unit displays on the display unit according to the change in the relative relationship detected by the relative relationship change detection unit. Prior to shifting the range of the rearview mirror-like image, the obstacle detected by the first obstacle detection unit may be included in the rearview mirror-like image displayed on the display unit. According to this, when an obstacle is included in the rear-view mirror-like image, the rear-view mirror-like image including the obstacle is continuously displayed on the display unit, thereby further preventing oversight of obstacles around the vehicle. Can do.

  In addition, as in claim 10, a second obstacle detection unit that detects an obstacle around the vehicle, and a second peripheral imaging unit other than the first peripheral imaging unit that images the periphery of the vehicle, and a display When the control unit detects an obstacle with the second obstacle detection unit, the control unit includes the obstacle detected by the second obstacle detection unit in preference to displaying the rearview mirror-like image on the display unit. The image captured by the second peripheral imaging unit may be displayed on the display unit. According to this, when an obstacle around the vehicle is detected by the second obstacle detection unit, the obstacle detected by the second obstacle detection unit has priority over the display of the rearview mirror-like image. To prevent obstacles around the vehicle from being overlooked.

  Further, as in claim 11, a vehicle reverse detection unit that detects the vehicle reverse is provided, and when the display control unit detects the vehicle reverse by the vehicle reverse detection unit, a rearview mirror that is displayed on the display unit The range of the like image may be shifted downward. According to this, it becomes easier for the driver to check the parking lot line during reverse parking, and the convenience is further improved.

1 is a block diagram showing a schematic configuration of an electronic mirror system 100. FIG. It is a schematic diagram for explaining the installation location and the imaging range of the first imaging device 1, the second imaging device 2, and the third imaging device 3. 6 is a schematic diagram for explaining a process of shifting the range of a rearview mirror-like image displayed on the display device 5. FIG. (A) is a schematic diagram which shows the relationship between the movement of a driver's viewpoint, and the shift | offset | difference of the range of the image reflected on the reflective surface of a rearview mirror, (b) is after a driver's viewpoint movement and the display apparatus 5 display. It is a schematic diagram which shows the relationship with the shift | offset | difference of the range of a mirror-like image. (A) is a schematic diagram which shows the relationship between the change of the inclination of the reflective surface of a room mirror, and the shift | offset | difference of the range of the image reflected on the reflective surface of a room mirror, (b) is a change of the inclination of the reflective surface of a room mirror, and 6 is a schematic diagram showing a relationship with a shift in a range of a rearview mirror-like image displayed on the display device 5. FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of an electronic mirror system 100 to which the present invention is applied. An electronic mirror system 100 illustrated in FIG. 1 is mounted on a vehicle, and includes a first imaging device 1, a second imaging device 2, a third imaging device 3, a first potentiometer 4, a display device 5, and a second potentiometer 6. , A shift position sensor 7 and a control device 8 connected thereto. The electronic mirror system 100 corresponds to the vehicle periphery monitoring system in the claims. Further, a vehicle equipped with the electronic mirror system 100 is hereinafter referred to as a host vehicle.

  The electronic mirror system 100 captures an image of the rear and rear side areas of the host vehicle reflected by the rearview mirror when the rearview mirror is installed in the host vehicle with the first imaging device 1 and images captured by the first imaging device 1. Is displayed on the display device 5 installed in the passenger compartment, and an image corresponding to the image displayed on the room mirror is presented to the driver.

  Here, for the sake of convenience, a description will be given by taking as an example a configuration in which an image corresponding to an image of an area projected on a room mirror is displayed on the display device 5, but the present invention is not necessarily limited thereto. For example, the present invention can be applied to an image of a region projected on a rear-view mirror other than the room mirror (that is, a door mirror or a fender mirror) in the same manner as the image of the region projected on the room mirror. That is, in the following description, a configuration may be adopted in which a door mirror or a fender mirror is replaced with a room mirror. Needless to say, when a door mirror or a fender mirror is replaced with a room mirror, other configurations are appropriately changed in accordance with the replacement.

  In addition, an image corresponding to an image of a region projected on a rearview mirror other than the room mirror (that is, a door mirror or a fender mirror) may be displayed on the display device 5 or a combination thereof. Furthermore, it is good also as a structure which displays the image corresponded to the image of the area | region projected on a different kind of rear-view mirror on one screen, and it is good also as a structure which each displays on several screens.

  The first imaging device 1 is configured by a known CCD camera or the like whose imaging range is set to a wide angle by, for example, a wide-angle lens, and is installed, for example, in the passenger compartment of the host vehicle. The second imaging device 2 is configured by a known CCD camera or the like whose imaging range is set to a wide angle by, for example, a wide-angle lens, and is installed outside the cabin of the host vehicle, for example. Further, the third image pickup device 3 is constituted by a known CCD camera or the like, and is installed in, for example, the passenger compartment of the host vehicle. The image information captured by the first imaging device 1, the second imaging device 2, and the third imaging device 3 is supplied to the control device 8.

  Here, the installation location and imaging range of the first imaging device 1, the second imaging device 2, and the third imaging device 3 will be described with reference to FIG. FIG. 2 is a schematic diagram for explaining the installation locations and imaging ranges of the first imaging device 1, the second imaging device 2, and the third imaging device 3. In addition, the arrow shown with the broken line in FIG. 2 has shown the optical axis of each camera of the 1st imaging device 1, the 2nd imaging device 2, and the 3rd imaging device 3. FIG.

  In the following, the forward direction of the host vehicle is forward, the backward direction is backward, the ceiling direction of the host vehicle is upward, the floor direction of the host vehicle is downward, the right hand side is rightward with respect to the forward direction, and the forward direction is forward. On the other hand, the left hand side is the left side.

  As shown in FIG. 2, the first imaging device 1 is attached to, for example, a passenger compartment ceiling behind the driver's seat and ahead of the rear window. The first imaging device 1 is installed so that the optical axis of the camera faces the rear window. And the 1st imaging device 1 images sequentially the area | region which spreads in the predetermined angle range behind the own vehicle. In detail, the 1st imaging device 1 images at least the range which can perform safety confirmation with the room mirror at the time of installing a room mirror in the own vehicle. In addition, the range in which the safety can be checked with the rearview mirror is the range in which the seat position of the driver's seat can be adjusted, the range in which the angle of the rearview mirror can be adjusted, and the height of the virtual driver (for example, the average height of a person) ) Etc. Hereinafter, an image captured by the first imaging device 1 is referred to as a vehicle rear image. Moreover, the 1st imaging device 1 is corresponded to the 1st periphery imaging part of a claim.

  In the present embodiment, the configuration in which the first imaging device 1 is attached to the vehicle compartment ceiling or the like behind the driver's seat and ahead of the rear window is not necessarily limited thereto. For example, the 1st imaging device 1 is good also as a structure attached to the vehicle body rear part outside a vehicle compartment, and imaging the area | region which spreads in a predetermined angle range behind the own vehicle.

  As shown in FIG. 2, the 2nd imaging device 2 shall be installed in the vehicle body rear part above the rear-part bumper of the own vehicle, for example. The second imaging device 2 is installed such that the optical axis of the camera faces the road surface at the rear of the vehicle body. And the 2nd imaging device 2 images sequentially the area | region which spreads in the predetermined angle range in the vicinity of the back of the own vehicle. Hereinafter, an image captured by the second imaging device 2 is referred to as a vehicle rear vicinity image. The second imaging device 2 corresponds to a second peripheral imaging unit in the claims.

  As shown in FIG. 2, the 3rd imaging device 3 shall be installed in front upper direction rather than a driver's seat, for example. The third imaging device 3 is installed so that the optical axis of the camera faces the virtual driver's face when it is assumed that the driver is seated on the driver's seat. Then, the third imaging device 3 sequentially captures the face of the driver sitting in the driver's seat. Further, the position of the virtual driver's face may be obtained based on the range in which the seat position of the driver's seat can be adjusted, the height of the virtual driver (for example, the average height of a person), and the like. Hereinafter, an image captured by the third imaging device 3 is referred to as a driver image. The third imaging device 3 corresponds to a driver imaging unit in claims.

  Returning to FIG. 1, the 1st potentiometer 4 is a sensor which detects the angle of the room mirror at the time of installing a room mirror in the own vehicle. The first potentiometer 4 may detect the vertical and horizontal angles with respect to the reference position. In the present embodiment, the angle detected by the first potentiometer 4 is handled as the reflection angle of the room mirror.

  The display device 5 is provided at a position where the driver can visually recognize the vehicle interior of the host vehicle, and sequentially displays images output from the control device 8. Therefore, the display device 5 corresponds to a display unit in claims. For example, the display device 5 can be configured using a liquid crystal display, an organic EL display, a plasma display, or the like. In addition, the display device 5 can change the orientation of a display surface (that is, a display surface) for displaying an image by at least one of manual and electric modes. The orientation of the display surface can be changed, for example, up, down, left, and right. Hereinafter, the description will be continued assuming that the display device 5 is provided, for example, in a dashboard.

  When the orientation of the display surface is changed manually, the orientation of the display surface may be changed by a physical external force from an occupant such as a driver. In addition, when the orientation of the display surface is changed electrically, according to the input received by the operation switch that accepts an input from the user to change the orientation of the display surface (for example, an input for instructing the direction of changing the orientation of the display surface) The direction of the display surface may be changed by operating driving means such as an electric motor for driving the display surface.

  In the present embodiment, the configuration in which the display device 5 is provided on the dashboard is shown, but the present invention is not necessarily limited thereto. For example, the display device 5 may be provided in a place other than the dashboard, or may be provided in a place where a room mirror is provided instead of the room mirror. Moreover, it is good also as a structure which utilizes the display provided in the vehicle-mounted navigation apparatus as the display apparatus 5. FIG. Further, a head-up display (HUD) that projects an image on a windshield (for example, a front window) may be configured as the display device 5. When the HUD is used as the display device 5, the windshield becomes the display surface of the display device 5.

  The second potentiometer 6 is a sensor that detects the angle of the display surface of the display device 5. Therefore, the 2nd potentiometer 6 is equivalent to the display surface direction detection part of a claim. The second potentiometer 6 may detect an angle with respect to a reference position in the vertical direction and the horizontal direction. In the present embodiment, the angle detected by the second potentiometer 6 is treated as the inclination of the display surface of the display device 5.

  The shift position sensor 7 is a sensor that detects the position of the shift position (or driving range) of the host vehicle, and outputs a signal indicating the detected position. When the host vehicle moves backward, the shift position sensor 7 detects that the shift position (or driving range) of the host vehicle has entered the reverse position (R). Therefore, the shift position sensor 7 corresponds to a vehicle reverse detection unit in claims.

  The control device 8 is configured as a normal computer, and includes a well-known CPU, ROM, RAM, I / O, and a bus line (all not shown) for connecting these configurations. The control device 8 executes various processes based on various information input from the first imaging device 1, the second imaging device 2, the third imaging device 3, the first potentiometer 4, the second potentiometer 6, and the shift position sensor 7. .

  Here, a schematic configuration of the control device 8 will be described. As shown in FIG. 1, the control device 8 includes a viewpoint related position detection unit 81, a relative relationship change detection unit 82, a first obstacle detection unit 83, a second obstacle detection unit 84, and a display control unit 85. .

  The viewpoint related position detection unit 81 detects a driver viewpoint related position that is a position of a part related to the viewpoint of the driver of the host vehicle from the driver image captured by the third imaging device 3. Therefore, the viewpoint related position detection unit 81 corresponds to the driver viewpoint related position detection unit in the claims. Here, the following description will be continued assuming that the viewpoint-related position detection unit 81 detects the position of the viewpoint of the driver of the host vehicle as the driver viewpoint-related position, for example. The driver viewpoint-related position may be represented by coordinates on a two-axis plane including, for example, a vertical axis and a horizontal axis.

  The driver viewpoint related position may be configured to detect a position other than the viewpoint (that is, the eye) as long as the position moves in conjunction with the movement of the position of the driver's viewpoint of the host vehicle. It may be configured to detect the position of the driver's face, the position of a part of the face other than the eyes, and the like.

  The viewpoint-related position detection unit 81 may also be configured to detect the position of the driver's viewpoint in the front-rear direction by image analysis of the driver image. In the present embodiment, the following description will be continued on the assumption that the viewpoint-related position detection unit 81 also detects the position of the driver's viewpoint in the front-rear direction.

  In the present embodiment, the configuration in which the driver viewpoint related position is detected from the driver image captured by the third imaging device 3 is shown, but the present invention is not necessarily limited thereto. If there is an apparatus capable of detecting the driver viewpoint related position other than the imaging apparatus, the apparatus may be configured to detect the driver viewpoint related position using the apparatus. For example, the driver viewpoint-related position may be detected by a triaxial acceleration sensor mounted on the driver's head.

  Based on the driver viewpoint related position detected by the viewpoint related position detection unit 81 and the inclination of the display surface of the display device 5 detected by the second potentiometer 6, the relative relationship change detection unit 82 A change in relative relationship with the driver viewpoint related position is detected.

  For example, the relative relationship change detection unit 82 holds in advance a reference value (hereinafter referred to as an inclination reference value) for the inclination of the display surface of the display device 5, and the display surface detected by the inclination reference value and the second potentiometer 6. And a change direction and a change amount of the display surface inclination from a position serving as a reference of the display surface inclination are detected. For example, the value when the display surface of the display device 5 is in a position where the tilt of the display surface does not occur in either the up-down direction or the left-right direction, that is, 0 for both the up-down direction and the left-right direction. And it is sufficient.

  In addition, the relative relationship change detection unit 82 holds a position (hereinafter referred to as a viewpoint related position reference value) serving as a reference for the driver viewpoint related position, and the driver detected by the viewpoint related position reference position and the viewpoint related position detection unit 81. The viewpoint-related position is compared, and the change direction and change amount of the driver viewpoint-related position from the position serving as the reference of the driver viewpoint-related position are detected. The driver viewpoint related position detected by the viewpoint related position detection unit 81 when the seat position of the driver seat is determined may be used as the viewpoint related position reference value. In addition, when the seat position of the driver's seat is determined, for example, it is detected that the parking brake has been released, and the shift position sensor 7 detects that the shift position of the host vehicle has entered the reverse position or the forward position. Therefore, the control device 8 may make a determination.

  The first obstacle detection unit 83 detects an obstacle such as a pedestrian from the vehicle rear image captured by the first imaging device 1. In addition, what is necessary is just to set it as the structure which detects the obstruction from a vehicle back image by the well-known image recognition which recognizes the object in an image using the dictionary for image recognition, for example. In this case, for example, a vehicle rear image using a dictionary (specifically, a cascade of boosted class discriminator with a haar-like feature characterized by a rectangular luminance difference) that has been machine-learned for a target obstacle in advance. It is only necessary to detect obstacles from.

  The second obstacle detection unit 84 detects an obstacle from the vehicle rear vicinity image imaged by the second imaging device 2. The second obstacle detection unit 84 may be configured to detect an obstacle from the vehicle rear vicinity image in the same manner as the first obstacle detection unit 83.

  The display control unit 85 performs a rearview mirror-like image display process for causing the display device 5 to display a rearview mirror-like image that is an image corresponding to an image displayed on the rear-view mirror of the vehicle. In the rearview mirror-like image display processing, for example, the display control unit 85 detects the angle detected by the first potentiometer 4 (that is, the reflection angle of the room mirror) and the viewpoint of the driver of the host vehicle detected by the viewpoint-related position detection unit 81. Based on the position, the vehicle rear image captured by the first imaging device 1 is converted into an image at the same viewpoint as the image projected on the room mirror of the host vehicle (that is, a rearview mirror-like image). As a method of converting the vehicle rear image captured by the first imaging device 1 into an image of the same viewpoint as the image displayed on the room mirror of the host vehicle, the captured image disclosed in Patent Document 1 is displayed on the door mirror. The image may be configured to be performed by a method similar to the method for converting the image to the same viewpoint as the image to be viewed.

  Then, the display control unit 85 cuts out a part of the rearview mirror-like image obtained by the rearview mirror-like image display process, and causes the display device 5 to display the cut-out rearview mirror-like image. Specifically, the first imaging device 1 uses a wide-angle lens to capture a wider range than the range of the image displayed on the vehicle's rear-view mirror, so that the image displayed on the vehicle's rear-view mirror is changed from the rear image of the vehicle. A region corresponding to at least a part is cut out, and a rearview mirror-like image corresponding to an image displayed on the room mirror is displayed on the display device 5.

  In addition, it is good also as a structure which cuts out the area | region containing at least the image projected on the rear-view mirror of the own vehicle, and displays it on the display apparatus 5. FIG. Moreover, it is good also as a structure which cuts out the area | region of the same range as the image projected on the rear-view mirror of the own vehicle, and displays it on the display apparatus 5. FIG. According to this, since the rear-view mirror-like image displayed on the display device 5 includes an image projected on the rear-view mirror of the host vehicle, the display device 5 is supported by the rear-view mirror (that is, an optical mirror). Not only can it be used as a substitute.

  Further, in the present embodiment, the rear of the vehicle imaged by the first imaging device 1 based on the angle detected by the first potentiometer 4 and the viewpoint position of the driver of the host vehicle detected by the viewpoint-related position detector 81. Although the configuration for converting an image into a rearview mirror-like image has been shown, the present invention is not necessarily limited thereto. For example, the position of the viewpoint of the driver of the host vehicle may be obtained based on the seat position of the driver's seat, the virtual height of the driver (for example, the average height of a person), and the like. In addition, the first potentiometer 4 is not provided, and the vehicle rearward image captured by the first imaging device 1 is rearranged based on a predetermined reflection angle of a virtual room mirror and the viewpoint position of the driver of the host vehicle. It is good also as a structure converted into a mirror-like image.

  Even when the vehicle is not provided with a room mirror and the display device 5 is used instead of the room mirror, the predetermined reflection angle of the virtual room mirror and the position of the viewpoint of the driver of the vehicle are determined. Basically, the vehicle rear image captured by the first imaging device 1 may be converted into a rearview mirror-like image. Also, the position of the driver's viewpoint of the host vehicle may be determined in advance based on the average seat position of the driver's seat or the average height of a person.

  Further, the display control unit 85 performs a display range moving process for shifting the range of the rearview mirror-like image displayed on the display device 5 in accordance with the change in the relative relationship detected by the relative relationship change detection unit 82. In the display range movement process, the display control unit 85 uses the change direction and change amount of the driver viewpoint related position detected by the relative relationship change detection unit 82 and the change direction and change amount of the tilt of the display surface of the display device 5. The range of the rearview mirror-like image displayed on the display device 5 is shifted.

  For example, the display control unit 85 determines the range of the rearview mirror-like image to be displayed on the display device 5 with respect to the change in the driver viewpoint related position detected by the relative relationship change detection unit 82 in any of the vertical and horizontal directions. The direction of the change is shifted in the opposite direction. As an example, when the change direction of the driver viewpoint related position detected by the relative change detection unit 82 is the right direction, the range of the rearview mirror-like image displayed on the display device 5 is shifted to the left. According to this, the range of the rearview mirror-like image displayed on the display device 5 can be shifted in the same manner as the shift of the range of the image reflected on the reflection surface of the room mirror.

  The display control unit 85 preferably shifts the range of the rearview-like image displayed on the display device 5 by an amount corresponding to the change amount of the driver viewpoint related position detected by the relative relationship change detection unit 82. Note that the extent to which the range of the rearview mirror-like image displayed on the display device 5 is shifted according to the change amount of the driver viewpoint related position depends on the image of the image reflected on the reflection surface of the room mirror with respect to the change amount of the driver viewpoint related position. In consideration of the shift of the range, it may be set to a level corresponding to the shift of the range of the image reflected on the reflection surface of the room mirror. According to this, the range of the rearview mirror-like image displayed on the display device 5 can be shifted in a manner more similar to the shift of the range of the image reflected on the reflection surface of the room mirror.

  Further, the display control unit 85 causes the display device 5 to display on the display device 5 in response to a change in the tilt of the display surface of the display device 5 detected in the relative relationship change detection unit 82 in any of the vertical and horizontal directions. The range of the image is shifted in the same direction as the direction of the change. As an example, when the change direction of the inclination of the display surface of the display device 5 detected by the relative change detection unit 82 is the right direction (that is, the direction in which the display surface is directed to the right), the display device 5 displays the change. Shift the range of the rearview mirror-like image to the right. According to this, the range of the rearview mirror-like image displayed on the display device 5 can be shifted in the same manner as the shift of the range of the image reflected on the reflection surface of the room mirror.

  Further, the display control unit 85 can shift the range of the rearview-like image displayed on the display device 5 by an amount corresponding to the amount of change in the tilt of the display surface of the display device 5 detected by the relative relationship change detection unit. preferable. It should be noted that how much the range of the rearview mirror-like image displayed on the display device 5 is shifted in accordance with the amount of change in the tilt of the display surface of the display device 5 depends on the amount of change in the tilt of the reflection surface of the room mirror. In consideration of the shift of the range of the image reflected on the reflection surface, it may be set to a level corresponding to the shift of the range of the image reflected on the reflection surface of the room mirror. According to this, the range of the rearview mirror-like image displayed on the display device 5 can be shifted in a manner more similar to the shift of the range of the image reflected on the reflection surface of the room mirror.

  When both the change in the driver viewpoint related position and the change in the tilt of the display surface of the display device 5 are detected by the relative relationship change detection unit 82, the rearview mirror-like state is changed according to the change amount in the driver viewpoint related position. The amount of shifting the range of the rearview mirror-like image in accordance with the amount of shifting the image range (hereinafter referred to as driver viewpoint-related position-dependent shift amount) and the amount of change in the tilt of the display surface of the display device 5 (hereinafter referred to as display surface tilt-dependent shift) The range of the rearview mirror-like image displayed on the display device 5 may be shifted in consideration of the amount).

  For example, the rearview mirror-like image range is shifted in accordance with the change amount of the driver viewpoint-related position (hereinafter referred to as the driver viewpoint-related position-dependent change direction) and the change amount of the inclination of the display surface of the display device 5. When the direction in which the range of the mirror-like image is shifted (hereinafter, the display surface tilt-dependent change direction) is the same direction, the display device is the sum of the driver viewpoint-related position-dependent shift amount and the display surface tilt-dependent shift amount. The range of the rearview mirror-like image displayed on the screen 5 may be shifted. Further, when the driver viewpoint related position dependent change direction and the display surface tilt dependent change direction are opposite, after the display device 5 displays only the difference between the driver viewpoint related position dependent shift amount and the display surface tilt dependent shift amount. What is necessary is just to set it as the structure which shifts the range of a mirror-like image.

  Here, the process of shifting the range of the rearview mirror-like image displayed on the display device 5 will be described with reference to FIG. FIG. 3 is a schematic diagram for explaining the process of shifting the range of the rearview mirror-like image displayed on the display device 5. A range A indicated by a broken line in FIG. 3 indicates a range of the rearview mirror-like image obtained by the rearview mirror-like image display process (that is, a range corresponding to the imaging range of the first imaging device 1), and is indicated by a solid line. A range B indicated by (2) indicates a range of the rearview mirror-like image that is actually displayed on the display device 5. Further, in FIG. 3, C indicates the upward direction, D indicates the downward direction, E indicates the right direction, and F indicates the left direction.

  When shifting the range of the rearview mirror-like image to be displayed on the display device 5 in the display range moving process of the display control unit 85, the range A to the range B of the rearview mirror-like image obtained by the rearview mirror-like image display process. The range of the rearview mirror-like image displayed on the display device 5 may be shifted by shifting the location where the image is cut out. For example, when the range of the rearview mirror-like image displayed on the display device 5 is shifted upward in the display range moving process, the location where the range B is cut out from the range A is shifted in the direction C. That's fine.

  If the direction of shifting the range of the rearview mirror-like image displayed on the display device 5 is downward, the cutout location may be shifted to the direction D. Further, when the direction in which the range of the rearview mirror-like image displayed on the display device 5 is shifted to the right, the cutout location may be shifted to the direction E. Furthermore, when the direction in which the range of the rearview mirror-like image displayed on the display device 5 is shifted to the left, the cutout location may be shifted to the F direction.

  When the first obstacle detection unit 83 detects an obstacle, the display control unit 85 gives priority to the display range movement process, and the obstacle detected by the first obstacle detection unit 83 is displayed on the display device 5. It may be included in the rearview mirror-like image displayed on the screen. For example, while the display control unit 85 does not perform the display range movement process and the first obstacle detection unit 83 detects an obstacle, a rearview mirror-like image including the obstacle is displayed on the display device 5. What is necessary is just composition.

  According to this, when an obstacle detected by the first obstacle detection unit 83 is included in the rearview mirror-like image displayed on the display device 5, the obstacle is detected by the first obstacle detection unit 83. While the object is detected, the rearview mirror-like image including the obstacle can be continuously displayed on the display device 5, and the obstacles around the host vehicle can be further prevented from being overlooked.

  In addition, when the second obstacle detection unit 84 detects an obstacle, the display control unit 85 has priority over displaying the rearview mirror-like image on the display device 5, and the second obstacle detection unit 84. The vehicle rear vicinity image imaged by the second imaging device 2 including the obstacle detected in (5) may be displayed on the display device 5. For example, while the second obstacle detection unit 84 detects an obstacle, the display control unit 85 switches to the display of a rearview mirror-like image, and the display control unit 85 displays the display of the vehicle rear vicinity image including the obstacle. What is necessary is just to make it the structure which the apparatus 5 performs.

  According to this, when an obstacle around the host vehicle is detected by the second obstacle detection unit 84, the vehicle rear vicinity image including the obstacle is displayed in preference to the display of the rearview mirror-like image. Displayed on the device 5 can further prevent oversight of obstacles around the host vehicle.

  In the present embodiment, a configuration in which the control device 8 includes the first obstacle detection unit 83 and the second obstacle detection unit 84 is shown. However, the configuration is not limited to this, and the control device 8 includes the first obstacle detection unit. 83 and the second obstacle detection unit 84 may not be included.

  Further, the display control unit 85 may be configured to shift the range of the rearview mirror-like image displayed on the display device 5 downward when the shift position sensor 7 detects the backward movement of the host vehicle. According to this, it becomes easier for the driver to check the parking lot line during reverse parking, and the convenience is further improved. In the present embodiment, the configuration in which the electronic mirror system 100 includes the shift position sensor 7 is shown. However, the configuration is not necessarily limited thereto, and the electronic mirror system 100 may not include the shift position sensor 7.

  In the electronic mirror system 100, display is performed according to a change in the relative relationship between the display surface of the display device 5 detected by the relative relationship change detection unit 82 and the driver viewpoint related position, which is the position of the portion related to the driver's viewpoint of the host vehicle. Since the range of the rearview mirror-like image displayed on the device 5 is shifted, when the driver moves the viewpoint by moving the body or head, the display is performed in the same manner as the shift of the range of the image displayed on the reflection surface of the room mirror. The range of the rearview mirror-like image displayed on the device 5 can be shifted.

  Further, when the driver moves the viewpoint by moving the body or head, the range of the rearview mirror-like image displayed on the display device 5 is shifted in the same manner as the shift of the range of the image reflected on the reflection surface of the rearview mirror. Thus, the same usability as that of the rearview mirror can be realized in the electronic mirror system. According to the above configuration, the usability of the electronic mirror system for the driver can be further improved.

  Here, the effect in this invention is demonstrated concretely using FIG. 4 (a)-FIG.5 (b). FIG. 4A is a schematic diagram showing the relationship between the movement of the driver's viewpoint and the shift of the range of the image shown on the reflection surface of the room mirror. FIG. 4B is a schematic diagram showing the relationship between the movement of the driver's viewpoint and the shift of the range of the rearview mirror-like image displayed on the display device 5. FIG. 5A is a schematic diagram showing the relationship between the change in the inclination of the reflection surface of the room mirror and the shift of the range of the image reflected on the reflection surface of the room mirror. FIG. 5B is a schematic diagram showing the relationship between the change in the inclination of the reflection surface of the rearview mirror and the shift of the range of the rearview mirror-like image displayed on the display device 5.

  In FIGS. 4A and 5A, G indicates a rear view mirror, and H in FIGS. 4A to 5B indicates a driver as viewed from above. . Also, I in FIG. 4A indicates the driver's line of sight before movement, and J indicates the driver's line of sight after movement. Further, K in FIG. 5A indicates the driver's line of sight before the change in the inclination of the reflection surface of the room mirror, and L indicates the line of sight of the driver after the change in the inclination of the reflection surface of the room mirror.

  Also, when the driver moves the body or head to move the viewpoint, the range of the rearview mirror-like image displayed on the display unit is shifted in the same way as the shift of the range of the image reflected on the reflection surface of the rearview mirror. As a result, the same usability as that of the rearview mirror can be realized in the vehicle periphery monitoring system. Therefore, according to the above configuration, the usability of the vehicle periphery monitoring system for the driver can be further improved. Become.

  It should be noted that in a rearview mirror composed of an optical mirror, the range of the image reflected on the reflection surface of the rearview mirror is that the driver's viewpoint of the host vehicle moves up / down / left / right relative to the reflection surface of the rearview mirror. The viewpoint moves in the opposite direction to the direction of movement. For example, as illustrated in FIG. 4A, when the viewpoint of the driver H moves to the right as indicated by a white arrow with respect to the reflection surface of the room mirror G, the driver H Since the incident angle and the reflection angle of the line of sight increase, the image reflected on the reflection surface of the room mirror G shifts to the left as indicated by the dashed arrow.

  In the electronic mirror system 100, a change in the viewpoint of the driver is detected based on the driver image picked up by the third imaging device 3, and the range of the rearview mirror-like image to be displayed on the display device 5 is changed in the change direction of the driver's viewpoint. Shift in the opposite direction. For example, as shown in FIG. 4B, when the viewpoint of the driver H moves to the right as indicated by the white arrow, the range of the rearview mirror-like image displayed on the display device 5 is indicated by the dashed arrow. Will shift to the left.

  Therefore, in the electronic mirror system 100, the range of the rearview mirror-like image displayed on the display device 5 can be shifted in the same manner as the shift of the range of the image reflected on the reflection surface of the room mirror. Therefore, according to the above configuration, usability of the electronic mirror system for the driver can be further improved.

  In the case of a rearview mirror consisting of an optical mirror, the range of the image reflected on the rearview mirror's reflection surface is such that the direction of the rearview mirror's reflection surface moves up, down, left and right with respect to the driver's viewpoint of the vehicle (that is, the room When the reflecting surface of the mirror is tilted up, down, left and right), the direction of the reflecting surface of the rearview mirror deviates in the same direction. For example, as shown in FIG. 5A, when the reflection surface of the room mirror G is directed leftward (tilted) as indicated by a solid arrow with respect to the viewpoint of the driver H, the driver with respect to the reflection surface of the room mirror G Since the incident angle and the reflection angle of the line of sight of H are increased, the image reflected on the reflection surface of the room mirror G is shifted to the left as indicated by the dashed arrow.

  In the electronic mirror system 100, based on the angle of the display surface of the display device 5 detected by the second potentiometer 6, a change in the orientation of the display surface of the display device 5 is detected and displayed on the display device 5. The range of the image is shifted in the same direction as the direction of change in the orientation of the display surface of the display device 5. For example, as shown in FIG. 5B, when the display surface of the display device 5 is directed leftward (tilted) as indicated by a solid arrow, the range of the rearview-like image displayed on the display device 5 is indicated by a broken line. It will be shifted to the left as indicated by the arrow.

  Therefore, in this respect, the electronic mirror system 100 can also shift the range of the rearview mirror-like image displayed on the display device 5 in the same manner as the shift of the range of the image reflected on the reflection surface of the room mirror. Therefore, even with the above configuration, the usability of the electronic mirror system for the driver can be further improved.

  In the above-described embodiment, the configuration is shown in which the range of the rearview mirror-like image displayed on the display device 5 is shifted by shifting the range to be cut out from the rearview mirror-like image obtained by the rearview mirror-like image display processing. However, this is not necessarily the case. For example, a driving unit such as a motor for driving the first imaging device 1 vertically and horizontally is provided, and the display control unit 85 controls the driving unit to move the imaging range of the first imaging device 1 vertically and horizontally to display The range of the rearview mirror-like image displayed on the device 5 may be shifted.

  Moreover, although the 2nd obstacle detection part 84 showed the structure which detects an obstacle from the vehicle back vicinity image imaged with the 2nd imaging device 2 in the above-mentioned embodiment, it does not necessarily restrict to this. For example, an obstacle may be detected based on a signal from an obstacle sensor such as an ultrasonic sonar or a millimeter wave radar that detects an obstacle within the imaging range of the second imaging device 2. In this case, when the second obstacle detection unit 84 detects an obstacle based on the signal from the obstacle sensor, the vehicle rear vicinity image captured by the second imaging device 2 by the display control unit 85 is displayed. What is necessary is just to set it as the structure displayed. Moreover, it is good also as a structure which provides a some obstruction sensor so that the detection range of an obstruction sensor may cover the imaging range of the 2nd imaging device 2. FIG.

  In addition, although the structure which displays the vehicle back vicinity image imaged with the 2nd imaging device 2 on the display apparatus 5 when the 2nd obstacle detection part 84 detected an obstacle was shown, the 2nd obstacle detection part 84 was shown. Until the obstacle is detected, the vehicle rear vicinity image captured by the second imaging device 2 may be displayed on a display device (not shown) other than the display device 5. Further, even after an obstacle is detected by the second obstacle detection unit 84, a vehicle rear vicinity image captured by the second imaging device 2 may be displayed on the display device other than the display device 5.

  In the above-described embodiment, the configuration in which the second imaging device 2 is installed in the rear part of the vehicle body and the area extending in the predetermined angular range in the vicinity of the rear of the host vehicle is shown, but this is not necessarily limited thereto. For example, it is good also as a structure which installs the 2nd imaging device 2 in places other than the vehicle body rear part, and images the area | region of a blind spot other than the vicinity of the back of the own vehicle. In addition, it is good also as a structure which installs two or more 2nd imaging devices 2, and images the area | region of various blind spots around the own vehicle. In this case, it is needless to say that an area that extends in the predetermined angular range in the vicinity of the rear of the host vehicle may be imaged.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the technical means disclosed in different embodiments can be appropriately combined. Such embodiments are also included in the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 1 1st imaging device (1st periphery imaging part), 2nd imaging device (2nd surrounding imaging part), 3rd 3rd imaging device (driver imaging part), 4 1st potentiometer, 5 display apparatus (display part), 6 second potentiometer (display surface direction detection unit), 7 shift position sensor (vehicle reverse detection unit), 8 control device, 81 viewpoint related position detection unit (driver viewpoint related position detection unit), 82 relative relationship change detection unit, 83 1st obstacle detection part, 84 2nd obstacle detection part, 85 Display control part, 100 Electronic mirror system (vehicle periphery monitoring system)

Claims (11)

Mounted on the vehicle,
A first peripheral imaging unit that images the periphery of the vehicle;
A display for displaying an image;
A display control unit for displaying on the display unit a rearview mirror-like image that is an image corresponding to an image displayed on the rearview mirror of the vehicle based on the image captured by the first peripheral imaging unit. A vehicle surrounding monitoring system,
A relative relationship change detection unit that detects a change in relative relationship between a display surface of the display unit and a driver viewpoint related position that is a position of a portion related to the viewpoint of the driver of the vehicle;
The vehicle periphery monitoring system, wherein the display control unit shifts a range of a rearview mirror-like image displayed on the display unit in accordance with a change in the relative relationship detected by the relative relationship change detection unit. In claim 1,
A driver imaging unit that images the driver of the vehicle;
A driver viewpoint-related position detection unit that detects the driver viewpoint-related position from an image of the driver imaged by the driver imaging unit;
The relative relationship change detection unit, as a change in the relative relationship between the display surface of the display unit and the driver viewpoint related position based on the driver viewpoint related position detected by the driver viewpoint related position detection unit, A vehicle periphery monitoring system that detects a change in the driver viewpoint related position with respect to the display surface of the vehicle. In claim 2,
The display control unit displays on the display unit the change in the driver viewpoint related position in any of the vertical and horizontal directions with respect to the display surface of the display unit detected by the relative relationship change detection unit. A vehicle periphery monitoring system, wherein the range of the rearview mirror-like image to be shifted is shifted in the direction opposite to the direction of the change. In claim 3,
The display control unit displays the display by an amount corresponding to the amount of change in the vertical or horizontal direction of the driver viewpoint related position with respect to the display surface of the display unit detected by the relative relationship change detection unit. A vehicle periphery monitoring system, characterized in that a range of a rearview mirror-like image displayed on a section is shifted in a direction opposite to the direction of the change. In any one of Claims 1-4,
The display surface of the display unit is capable of changing direction by at least one of manual and electric,
A display surface orientation detection unit for detecting the orientation of the display surface;
The relative relationship change detection unit is configured to detect the driver viewpoint as a change in a relative relationship between the display surface of the display unit and the driver viewpoint related position based on the orientation of the display surface detected by the display surface direction detection unit. A vehicle periphery monitoring system that detects a change in the orientation of the display surface of the display unit relative to a related position. In claim 5,
The display control unit responds to a change in the direction of the display surface of the display unit with respect to the driver viewpoint related position detected by the relative change detection unit in any one of up, down, left, and right directions. A vehicle periphery monitoring system, wherein a range of a rearview mirror-like image displayed on the vehicle is shifted in the same direction as the direction of the change. In claim 6,
The display control unit is an amount corresponding to the amount of change in the direction of the display surface of the display unit with respect to the driver viewpoint related position detected by the relative relationship change detection unit in any of the upper, lower, left and right directions, A vehicle periphery monitoring system, wherein a range of a rearview mirror-like image displayed on the display unit is shifted in the same direction as the direction of the change.   The vehicle viewpoint monitoring system, wherein the driver viewpoint related position is any one of a face and eyes of a driver of the vehicle. In any one of Claims 1-8,
A first obstacle detection unit that detects an obstacle around the vehicle from an image captured by the first peripheral imaging unit;
The display control unit displays on the display unit according to the change in the relative relationship detected by the relative relationship change detection unit when the obstacle is detected by the first obstacle detection unit. Prior to shifting the range of the like image, the obstacle detected by the first obstacle detecting unit is included in the rearview mirror-like image to be displayed on the display unit. Monitoring system. In any one of Claims 1-9,
A second obstacle detector for detecting obstacles around the vehicle;
A second peripheral imaging unit other than the first peripheral imaging unit that images the periphery of the vehicle,
In the case where the second obstacle detecting unit detects the obstacle, the display control unit has priority over displaying the rearview mirror-like image on the display unit, and the second obstacle detecting unit. A vehicle periphery monitoring system, wherein an image picked up by the second peripheral image pickup unit including an obstacle detected in (2) is displayed on the display unit. In any one of Claims 1-10,
A vehicle reverse detection unit for detecting the reverse of the vehicle;
The display control unit shifts a range of a rearview mirror-like image to be displayed on the display unit downward when the vehicle reverse detection unit detects the reverse of the vehicle. system.

Images