JP2008213649A - Periphery monitoring device for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display an image of a driver's blind spot in a mode without making the driver feel odd. <P>SOLUTION: The periphery monitoring device for a vehicle is equipped with a peripheral environment monitoring camera 10 for imaging the outside of a vehicle, a camera direction control part 34 for changing an imaging direction of the peripheral environment monitoring camera 10, a display 44 or 49 for displaying an image captured by the peripheral environment monitoring camera 10, a driver monitoring camera 20 for imaging a driver and a sight line direction detecting part 32 for detecting a sight line direction of a driver from the image result of the driver monitoring camera 20, wherein a camera direction control part 34 changes the imaging direction of the peripheral environment monitoring camera 10 so as to capture an image at the dead angle following the change when a sight line of a driver detected by the sight line direction detecting part 32 is changed to a direction of display 44 or 49. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a vehicle periphery monitoring device that displays an image that covers a blind spot of a driver.

Conventionally, a vehicle surroundings visual recognition provided with a vehicle surroundings visual recognition device having an imaging means for capturing an ambient situation of the vehicle and an external image area acquisition means for acquiring the ambient situation imaged by the imaging means as external image area information. In the system, the vehicle periphery visual recognition device includes an image display monitor fitted in a pillar portion of the vehicle, a visual line position detection unit that detects a visual line position by a driver of the vehicle, and the visual line position detection unit. Based on the detected line-of-sight position, out of the image area information acquired by the imaging means, a blind spot image area extracting means for extracting a blind spot image area that becomes a blind spot visual field of the driver, and the image display monitor, 2. Description of the Related Art A vehicle surrounding visual recognition system including an image display control unit that performs control to display a blind spot image region is known (see, for example, Patent Document 1).
JP 2005-125828 A

  However, in the invention described in Patent Document 1, since the outside scenery image corresponding to the driver's line-of-sight direction is clipped and displayed using a wide-angle fixed camera provided outside the car, the image displayed on the monitor is actually This is inconsistent with the scenery outside the window when viewed by the driver, and may cause the driver to feel uncomfortable.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a vehicle periphery monitoring device that can display an image of a driver's blind spot in an uncomfortable manner.

In order to achieve the above object, a vehicle periphery monitoring apparatus according to a first aspect of the present invention includes first imaging means for imaging the outside of a vehicle,
Imaging direction control means for changing the imaging direction of the imaging means;
Display means provided on a vehicle member forming a blind spot of a driver of the vehicle and displaying an image captured by the first imaging means;
A second imaging means for imaging the driver;
Gaze direction detecting means for detecting the gaze direction of the driver based on the imaging result of the second imaging means,
When the driver's line-of-sight direction detected by the line-of-sight direction detecting unit changes in the direction of the display unit, the imaging direction control unit follows the change and captures the blind spot image. The imaging direction of the first imaging unit is changed.

A second invention is the vehicle periphery monitoring apparatus according to the first invention,
The first imaging means is attached in the vicinity of the vehicle member.

A third aspect of the invention is the vehicle periphery monitoring device according to the first aspect of the invention,
2. The vehicle periphery monitoring device according to claim 1, wherein the display unit is activated when the driver's line-of-sight direction detected by the line-of-sight direction detection unit changes in the direction of the display unit.

4th invention is the vehicle periphery monitoring apparatus which concerns on 1st invention,
The image processing apparatus further includes a display image generation unit configured to cut out a part of the image captured by the first imaging unit with a size corresponding to a screen size of the display unit and generate a display image displayed on the display unit. It is characterized by.

A fifth aspect of the present invention is the vehicle periphery monitoring apparatus according to the first aspect,
The display image generation unit generates the display image by cutting out the image portion of the blind spot from the image captured by the first imaging unit.

A sixth invention is the vehicle periphery monitoring apparatus according to the first invention,
The vehicle member may be at least one of a pillar, a door, a dashboard, and a roof.

A seventh invention is the vehicle periphery monitoring apparatus according to the first invention,
The vehicle member is a roof;
The display unit is activated when the driver's line-of-sight direction detected by the line-of-sight direction detection unit is changed to the direction of the display unit.

  ADVANTAGE OF THE INVENTION According to this invention, the periphery monitoring apparatus for vehicles which can display a driver | operator's blind spot image in a mode without a sense of incongruity is obtained.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing an embodiment of a vehicle periphery monitoring apparatus 1 according to the present invention. The vehicle periphery monitoring device 1 of the present embodiment is configured with a periphery monitoring ECU 30 as a center. The peripheral monitoring ECU 30 is configured as a microcomputer including a CPU, a ROM, a RAM, and the like that are connected to each other via a bus (not shown).

  A peripheral environment monitoring camera 10, motors 12 and 14, a driver monitoring camera 20, and a display device 40 are connected to the peripheral monitoring ECU 30 via an appropriate communication line such as a CAN (controller area network). Yes.

  The surrounding environment monitoring camera 10 captures an image of the external environment of the vehicle by using a lens and an image sensor such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). During operation, the surrounding environment monitoring camera 10 acquires an image of the external environment of the vehicle (hereinafter referred to as “external environment image”) in real time and supplies the image to the surrounding monitoring ECU 30 in a stream format of a predetermined frame period (for example, 30 fps). It may be. The surrounding environment monitoring camera 10 is mounted on the front portion of the vehicle so as to be rotatable (or adjustable in orientation) with respect to the vehicle body. In this example, the surrounding environment monitoring camera 10 is provided at the center of the vehicle (for example, the upper edge of the front windshield near the rearview mirror, etc.) because it is shared by a plurality of displays 44 to 49 as will be described later. . The rotation of the surrounding environment monitoring camera 10 is realized by actuators such as motors 12 and 14.

  The degree of freedom of rotation of the surrounding environment monitoring camera 10 may be only rotation about one axis in the substantially vertical direction so that the direction can be adjusted in the left-right direction in front of the vehicle, or the direction adjustment in the height direction. In other words, the rotation may be about two axes in the horizontal direction and the substantially vertical direction, and further about three axes. Here, the degree of freedom of rotation of the surrounding environment monitoring camera 10 is rotation about two axes in the lateral direction and the substantially vertical direction, and is assumed to be realized by the motors 12 and 14, respectively. In other words, the horizontal direction of the surrounding environment monitoring camera 10 can be adjusted by driving the motor 12, and the vertical direction of the surrounding environment monitoring camera 10 can be adjusted by driving the motor 14.

  The motors 12 and 14 may be servo motors such as DC motors and brushless motors, for example. The motors 12 and 14 are driven under the control of the periphery monitoring ECU 30. The control may be a semi-closed loop or a full semi-closed loop.

  The driver monitoring camera 20 is a camera including, for example, a color or infrared sensitive CCD (charge-coupled device) sensor array. The driver monitoring camera 20 is provided at an appropriate location of the vehicle so that, for example, the front surface of the driver (for example, the face portion from the front) can be captured. For example, the driver monitoring camera 20 is installed at an appropriate location such as a vehicle rearview mirror, a dashboard of an instrument panel, or a steering column. During operation, the driver monitoring camera 20 acquires an image of the driver's face in real time (hereinafter referred to as “face image”) and supplies it to the surroundings monitoring ECU 30 in a stream format of a predetermined frame period (for example, 30 fps). It may be.

  The display device 40 is provided on a vehicle member that forms a blind spot of the driver of the vehicle. As an example, the display device 40 includes six displays 44 to 49. Here, the vehicle members that form the blind spot are, for example, the left A pillar, the right A pillar, the roof, the dashboard, the left door, and the right door of the vehicle.

  The displays 44 and 45 are attached to the left and right A pillars of the vehicle, respectively. The displays 44 and 45 are preferably mounted in such a form that they do not form a new blind spot. From this point of view, the displays 44 and 45 preferably each have a size and shape along the corresponding pillar shape. The displays 44 and 45 are preferably sheet-type displays such as electronic paper and electronic screens. Hereinafter, the displays 44 and 45 are also referred to as a left pillar display 44 and a right pillar display 45, respectively.

  The display 46 is attached to the roof of the vehicle, and may be a large display replacing the sunroof, for example. Similarly, the display 46 is preferably a sheet-type display such as an electronic paper or an electronic screen from the viewpoint of space. Hereinafter, the display 46 is also referred to as a roof display 46.

  The display 47 is attached to the dashboard of the vehicle. Similarly, from the viewpoint of space, the display 47 is preferably a sheet-type display such as an electronic paper or an electronic screen. Hereinafter, the display 47 is also referred to as a dashboard display 46.

  The displays 48 and 49 are respectively attached to the left and right doors of the vehicle. Similarly, the displays 48 and 49 are preferably sheet-type displays such as electronic paper and electronic screen from the viewpoint of space. Hereinafter, the displays 48 and 49 are also referred to as a left door display 44 and a right door display 45, respectively.

  These displays 44 to 49 operate under the control of the periphery monitoring ECU 30. The displays 44 to 49 may be in an off state in a normal state, and are activated when an on command is received from the surrounding monitoring ECU 30 and are turned on.

  The peripheral monitoring ECU 30 includes a gaze direction detection unit 32, a camera direction control unit 34, and a display control unit 36 as main functional units.

  The line-of-sight direction detection unit 32 recognizes the direction of the driver's face through image processing on the face image acquired by the driver monitoring camera 20, and based on the recognized face direction, the direction of the driver's line of sight (line of sight) Information on the direction (hereinafter also referred to as “line-of-sight information”) is generated.

  For example, the line-of-sight direction detection unit 32 first extracts representative points (feature points) of the face from the face image acquired by the driver monitoring camera 20. This feature extraction method may be any appropriate method, and for example, a technique based on Active Appearance Model (AAM) may be used. An appropriate edge detection algorithm (eg, Sobel's edge detection algorithm) is then applied to extract the boundary between the face and facial features. Next, using the edges and feature points extracted as described above, the facial parts are separated as shown in FIG. 2, and the shape of the facial parts is extracted. Note that FIG. 2 shows only a part of the face as a representative, and shows an image in which the eyebrows and eyes of the face are extracted as parts. The line-of-sight direction detection unit 32 detects the position or orientation of the extracted facial parts and each of the pre-stored postures (for example, the posture when viewing each of the displays 44 to 49, or the posture looking at the front. The current posture (face orientation) is detected by comparing the degree of matching with the position or orientation of the same part. The orientation of the face may be represented by, for example, a rotation angle about three axes with the front direction of the face in a normal posture as one axis. The gaze direction detection unit 32 determines the driver's gaze direction and generates gaze information under the assumption that the driver's face direction corresponds to the driver's gaze direction (gaze direction). . Note that there are various gaze method detection methods by image recognition processing in the gaze direction detection unit 32, and the present invention is not limited to the above-described one. For example, the line-of-sight direction detection unit 32 extracts the outline of the eyeball as a face part, and based on the position of the eyeball in the outline of the eye and the driver's face orientation recognized as described above, May be detected. The line-of-sight information generated in this way is supplied to the display control unit 36 and the camera orientation control unit 34.

  The camera direction control unit 34 monitors the surrounding environment via the motors 12 and 14 so that the surrounding environment monitoring camera 10 faces in a direction corresponding to the driver's line-of-sight direction based on the line-of-sight information from the line-of-sight direction detection unit 32. The direction of the camera 10 (imaging direction) is controlled. For example, as shown in FIG. 3, when the driver's line-of-sight direction is the direction A1 when viewing the left pillar display 44, the camera direction control unit 34 changes the direction of the surrounding environment monitoring camera 10 to the direction A1. Change to parallel direction B1. Also, as shown in FIG. 3, when the driver's line-of-sight direction is the direction A2 when viewing the right pillar display 45, the camera direction control unit 34 changes the direction of the surrounding environment monitoring camera 10 to the direction A2. Change in parallel direction B2.

  The example shown in FIG. 3 shows an example of adjusting the orientation of the surrounding environment monitoring camera 10 in two dimensions within the plane, but it is desirable that the orientation of the surrounding environment monitoring camera 10 be adjusted in three dimensions. That is, the camera orientation control unit 34 controls the rotation angle of the motor 12 so that the orientation of the surrounding environment monitoring camera 10 is parallel to the driver's line-of-sight direction in the three-dimensional space, and The direction of the direction (rotation angle around the X1 axis in the figure) is adjusted, and the rotation angle of the motor 14 is controlled to set the vertical direction of the surrounding environment monitoring camera 10 (rotation angle around the X2 axis in the figure). It is desirable to adjust.

  The display control unit 36 generates a display image (video) to be displayed on the displays 44 to 49 based on the external environment image supplied from the surrounding environment monitoring camera 10. Based on the line-of-sight information from the line-of-sight direction detection unit 32, the display control unit 36 has a display ahead of the driver's line-of-sight direction detected by the line-of-sight direction detection unit 32 described later, among the displays 44 to 49. The generated display image is supplied to display the display image.

  FIG. 4 is a diagram illustrating a method for generating a display image to be displayed on the left pillar display 44 as an example. FIG. 4A illustrates an external environment image (original image) from the surrounding environment monitoring camera 10. 4 (B) is a diagram showing a cutout region 80 for the external environment image from the surrounding environment monitoring camera 10, and FIG. 4 (C) shows a display image generated based on the external environment image of FIG. 4 (A). FIG.

  The display control unit 36 first determines the position of the cutout region 80 with respect to the external environment image from the surrounding environment monitoring camera 10 based on the line-of-sight information from the line-of-sight direction detection unit 32, as shown in FIG. The shape of the cutout region 80 is preferably set to be similar to the shape of the left pillar display 44. The size (scaling factor) of the cutout region 80 may be fixed, but is variable according to the position of the driver's viewpoint (eyes) that can be detected by the line-of-sight direction detection unit 32 and the distance from the left A pillar. May be. The cut-out area 80 in the external environment image (original image) can be detected by the line-of-sight direction detection unit 32 so as to correspond to a portion where the driver is not visible due to the left pillar in the direction of the driver's line of sight (that is, a blind spot portion). Positioning is performed based on the relative relationship between the position of the viewpoint of the driver and the position of the viewpoint (imaging device) of the surrounding environment monitoring camera 10. When the position of the cutout region 80 is determined in this way, the display control unit 36 cuts out (extracts) the image portion in the cutout region 80 to generate a display image as shown in FIG. To do. The display image may be further adjusted in scaling, brightness, or the like according to the resolution of the display to be output.

  FIG. 5A shows a state before the display image generated in FIG. 4C is displayed on the left pillar display 44, and FIG. 5B shows the display generated in FIG. 4C. It is a figure which shows the state by which the image was displayed on the left pillar display.

  By looking at the display image shown in FIG. 5B, the driver can feel the external environment of the area where the left pillar is blind spot with the feeling that the left pillar does not exist (the feeling of seeing through the left pillar). I can grasp it. As a result, the driver can visually recognize a useful external environment that covers the driver's blind spot, and the driving safety of the vehicle is improved. For example, in the example shown in the figure, the driver notices the presence of a pedestrian that could not be seen with the left pillar by looking at the display image shown in FIG. 5B, and drives while paying attention to the pedestrian. Can do.

  Thus, according to the present embodiment, as described above, the direction of the surrounding environment monitoring camera 10 is changed following the change in the driver's line-of-sight direction detected by the line-of-sight direction detection unit 32, and the driver Since the external environment image from the surrounding environment monitoring camera 10 is displayed on the previous display in the line-of-sight direction, a display image without a sense of incongruity captured in an image-capturing direction suitable for the driver in the line-of-sight direction is displayed in the driver's line-of-sight It can be displayed on the corresponding display. As a result, the driver can grasp the blind spot without a sense of incongruity.

  Next, a specific example of main processing executed by the periphery monitoring ECU 30 will be described with reference to FIG. Here, for convenience of explanation, a case will be described in which the display device 40 is three displays, a left pillar display 44, a right pillar display 45, and a roof display 46. The processing routine shown in FIG. 6 is repeatedly executed at predetermined intervals until the ignition switch of the vehicle is turned on and turned off.

  In step 100, the gaze direction detection unit 32 detects the gaze direction of the driver based on the face image obtained from the driver monitoring camera 20 in the current cycle.

  In step 102, the gaze direction detection unit 32 determines whether or not the driver's gaze direction detected in the current cycle has changed by a predetermined angle or more from the driver's gaze direction detected in the previous cycle. The predetermined angle may be a large value that can detect a change in the line-of-sight direction from the right pillar display 45 direction to the left pillar display 44 direction, for example, but here, for example, the line-of-sight direction within the same right pillar display 45 direction It is set to a relatively small value so that a slight change can be detected. If the driver's line-of-sight direction has changed by a predetermined amount or more, the process proceeds to step 108. If the driver's line-of-sight direction has not changed by a predetermined amount or more, the routine proceeds to step 104.

  In step 104, the display control unit 36 determines whether any one of the displays 45, 46, and 47 is currently on. If any one of the displays 45, 46, and 47 is in the on state, the process proceeds to step 106. If all the displays 45, 46, and 47 are in the off state, the processing routine for the current cycle is terminated as it is. .

  In step 106, the display control unit 36 generates a display image based on the external environment image from the surrounding environment monitoring camera 10 obtained in the current cycle, and the generated display image is displayed on the display (45, 46 or 47). As a result, the display image is displayed on the display (45, 46, or 47) that is currently on. In this case, in the display control unit 36, the external environment image used to generate the display image is an external environment image captured by the surrounding environment monitoring camera 10 in the same direction as the previous cycle.

  In step 108, the gaze direction detection unit 32 determines whether or not the driver's gaze direction detected in the current cycle is the left pillar direction (the left pillar display 44 direction). If the driver's line-of-sight direction detected in the current cycle is the left pillar direction, the process proceeds to step 110, and otherwise, the process proceeds to step 120.

  In step 110, the control target value of the motors 12 and / or 14 is set in the camera orientation control unit 34 so that the imaging direction of the surrounding environment monitoring camera 10 is parallel to the line-of-sight direction of the driver detected in the current cycle. It is determined. Then, rotation angle control of the motors 12 and / or 14 according to the determined control target value is executed. As a result, the imaging direction of the surrounding environment monitoring camera 10 is adjusted in parallel with the driver's line-of-sight direction detected in the current cycle.

  In step 112, the display control unit 36 determines whether or not the left pillar display 44 is currently in an on state. If the left pillar display 44 is in the on state, that is, if the driver's line-of-sight direction is also in the left pillar direction in the previous cycle, the process proceeds to step 116. Otherwise, the process proceeds to step 114.

  In step 114, the display controller 36 generates and outputs an ON command for the left pillar display 44. As a result, the left pillar display 44 is activated.

  In step 116, the display control unit 36 generates a display image based on the external environment image captured by the surrounding environment monitoring camera 10 whose orientation has been adjusted in step 110.

  In step 118, the display image generated in step 116 is supplied to the left pillar display 44 in the display control unit 36. As a result, the display image generated in step 116 is displayed on the left pillar display 44. In the case of going through step 114, the start timing of the left pillar display 44 may be adjusted so that a display image is output simultaneously with the start of the start of the left pillar display 44.

  In step 120, the gaze direction detection unit 32 determines whether or not the driver's gaze direction detected in the current cycle is the right pillar direction (right pillar display 45 direction). If the driver's line-of-sight direction detected in the current cycle is the right pillar direction, the process proceeds to step 122; otherwise, the process proceeds to step 132.

  In step 122, the camera target control unit 34 sets the control target values of the motors 12 and / or 14 so that the imaging direction of the surrounding environment monitoring camera 10 is parallel to the driver's line-of-sight direction detected in the current cycle. It is determined. Then, rotation angle control of the motors 12 and / or 14 according to the determined control target value is executed. As a result, the imaging direction of the surrounding environment monitoring camera 10 is adjusted in parallel with the driver's line-of-sight direction detected in the current cycle.

  In step 124, it is determined whether or not the right pillar display 45 is currently on. If the right pillar display 45 is in the on state, that is, if the driver's line-of-sight direction is also in the right pillar direction in the previous cycle, the process proceeds to step 128. Otherwise, the process proceeds to step 126.

  In step 126, the display control unit 36 generates and outputs an ON command for the right pillar display 45. As a result, the right pillar display 45 is activated.

  In step 128, the display control unit 36 generates a display image based on the external environment image captured by the surrounding environment monitoring camera 10 whose orientation is adjusted in step 122.

  In step 130, the display image generated in step 128 is supplied to the right pillar display 45 in the display control unit 36. As a result, the display image generated in step 128 is displayed on the right pillar display 45.

  In step 132, the gaze direction detection unit 32 determines whether or not the gaze direction of the driver detected in the current cycle is the roof direction (the direction of the roof display 46). If the driver's line-of-sight direction detected in the current cycle is the roof direction, the process proceeds to step 138; otherwise, the process proceeds to step 134.

  In step 134, the display control unit 36 determines whether any one of the displays 45, 46, and 47 is currently in an on state. If any one of the displays 45, 46, and 47 is in the on state, the process proceeds to step 136. If all the displays 45, 46, and 47 are in the off state, the processing routine for the current cycle is terminated as it is. .

  In step 136, the display control unit 36 generates and outputs an off command for the display (45, 46, or 47) that is currently on. As a result, the display (45, 46, or 47) that is currently on is turned off, and all the displays 45 to 47 are turned off.

  In step 138, the camera target control unit 34 sets the control target values of the motors 12 and / or 14 so that the imaging direction of the surrounding environment monitoring camera 10 is parallel to the driver's line-of-sight direction detected in the current cycle. It is determined. Then, rotation angle control of the motors 12 and / or 14 according to the determined control target value is executed. As a result, the imaging direction of the surrounding environment monitoring camera 10 is adjusted in parallel with the driver's line-of-sight direction detected in the current cycle.

  In step 140, it is determined whether or not the roof display 46 is currently on. When the roof display 46 is in the on state, that is, when the driver's line-of-sight direction is also the roof direction in the previous cycle, the process proceeds to step 144. Otherwise, the process proceeds to step 142.

  In step 142, the display control unit 36 generates and outputs an ON command for the roof display 46. As a result, the roof display 46 is activated.

  In step 144, the display control unit 36 generates a display image based on the external environment image captured by the surrounding environment monitoring camera 10 whose direction is adjusted in step 138.

  In step 146, the display image generated in step 144 is supplied to the roof display 46 in the display control unit 36. As a result, the display image generated in step 144 is displayed on the roof display 46.

  According to the present embodiment described above, the following excellent effects can be obtained.

  First, as described above, the direction of the surrounding environment monitoring camera 10 is changed following the change in the driver's line-of-sight direction, so that, for example, compared with a configuration using a wide-angle lens (fisheye lens), A display image without a sense of incongruity captured in a suitable imaging direction can be displayed. As a result, the driver can grasp the blind spot without a sense of incongruity.

  Also, since the direction of the surrounding environment monitoring camera 10 is changed only when the driver's line of sight is in the direction of the left pillar display 44 or the like, the followability of the line of sight of the surrounding environment monitoring camera 10 to the movement of the driver's line of sight Can be secured. In addition, when the driver's line-of-sight direction is not in the display direction, the surrounding environment monitoring camera 10 can be used for other purposes (for example, used as a white line recognition camera).

  When the roof display 46 is provided instead of the sunroof, an image of the sky outside is displayed on the roof display 46 when the driver looks up at the ceiling, so the sunroof is opened to the driver. It can give a visual refreshing feeling. From this point of view, the roof display 46 may be activated and output a display image when the line-of-sight direction of an occupant other than the driver (for example, a passenger on the passenger seat) is the ceiling direction, and the seat reclines. When it is tilted to the position, the activation and the display image may be output.

  The following modifications can be considered for the embodiment described above.

  For example, in the above-described embodiment, the surrounding environment monitoring camera 10 is configured to be rotatable around two axes. Therefore, by sharing the surrounding environment monitoring camera 10, display images to be displayed on the various displays 44 to 49. The original image of has been acquired. However, a dedicated ambient environment monitoring camera may be set for a certain display. For example, with respect to the left pillar display 44, the dashboard display 47, and the left door display 48, a first ambient environment monitoring camera provided on the left front portion of the vehicle (for example, outside the vehicle such as a left door mirror or inside the left pillar). For the right pillar display 45 and the right door display 49, a second ambient environment monitoring camera provided on the right front portion of the vehicle (for example, outside the vehicle such as a right door mirror or inside the right pillar) For the display 46, a third surrounding environment monitoring camera provided at the top of the vehicle may be used.

  Further, the surrounding environment monitoring camera 10 is configured by a camera having a zoom function capable of changing the magnification, and the distance between the position of the driver's viewpoint (eyes) and the position of the display device 40 that can be detected by the line-of-sight direction detection unit 32. The magnification of the surrounding environment monitoring camera 10 may be finely adjusted so as to follow the change of the image, and an image obtained from the surrounding environment monitoring camera 10 with the magnification adjusted may be displayed on the display device 40. For example, when the driver's line-of-sight direction is the left A pillar direction (the left pillar display 44 direction), the magnification of the surrounding environment monitoring camera 10 is determined by the position of the driver's viewpoint (eyes) and the left A pillar (the left pillar display). 44).

  In the embodiment described above, the “first imaging means” in the appended claims is realized by the surrounding environment monitoring camera 10, and the “second imaging means” in the claims is the driver. The “imaging direction control means” in the scope of the patent claims is realized by the camera orientation control unit 34 of the periphery monitoring ECU 30, and the “display means” in the scope of the claims is the display device 40. The “line-of-sight direction detecting means” is realized by the displays 44 to 49, and is realized by the line-of-sight direction detecting unit 32 of the periphery monitoring ECU 30.

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the present invention. Can be added.

It is a block diagram which shows one Example of the vehicle periphery monitoring apparatus 1 by this invention. It is a figure which shows the image from which the eyebrows and eye part of the face were extracted as parts. It is a figure which shows the example of control of the direction of the environment monitoring camera 10 by the camera direction control part 34, and is a figure which shows roughly the relationship between a driver | operator's gaze direction and the direction of the camera direction control part 34 by a top view. It is a figure which shows the production | generation method of the display image which should be displayed on the left pillar display. FIG. 5 is a diagram showing a state where the display image generated in FIG. 4C is displayed on the left pillar display 44. 3 is a flowchart showing a flow of main processing executed by a peripheral monitoring ECU 30.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle periphery monitoring apparatus 10 Surrounding environment monitoring camera 14 Motor 12 and 20 Driver monitoring camera 30 Perimeter monitoring ECU
32 Gaze direction detection unit 34 Camera direction control unit 36 Display control unit 40 Display device 44 Left pillar display 45 Right pillar display 46 Roof display 47 Dashboard display 48 Left door display 49 Right door display 80 Cutout area

Claims (7)

First imaging means for imaging the exterior of the vehicle;
Imaging direction control means for changing the imaging direction of the imaging means;
A display unit provided on a vehicle member forming a blind spot of a driver of the vehicle, and displaying an image captured by the first imaging unit;
A second imaging means for imaging the driver;
Gaze direction detecting means for detecting the gaze direction of the driver based on the imaging result of the second imaging means,
When the driver's line-of-sight direction detected by the line-of-sight direction detecting unit changes in the direction of the display unit, the imaging direction control unit follows the change and captures the blind spot image. A vehicle periphery monitoring device, wherein the imaging direction of the first imaging means is changed.   The vehicle periphery monitoring device according to claim 1, wherein the first imaging unit is attached in the vicinity of the vehicle member.   The vehicle periphery monitoring device according to claim 1, wherein the display unit is activated when the driver's line-of-sight direction detected by the line-of-sight direction detection unit changes in the direction of the display unit.   A display image generating unit that generates a display image displayed on the display unit by cutting out a part of an image captured by the first imaging unit in a size corresponding to a screen size of the display unit; The vehicle periphery monitoring apparatus according to claim 1.   2. The vehicle periphery monitoring device according to claim 1, wherein the display image generation unit generates the display image by cutting out the blind spot image portion of the image captured by the first imaging unit.   The vehicle periphery monitoring device according to claim 1, wherein the vehicle member is at least one of a pillar, a door, a dashboard, and a roof. The vehicle member is a roof;
2. The vehicle periphery monitoring device according to claim 1, wherein the display unit is activated when the driver's line-of-sight direction detected by the line-of-sight direction detection unit changes in the direction of the display unit.

Images