JP2008013023A - Inner mirror device and method of adjusting angle thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an inner mirror device capable of automatically and precisely adjusting the angle of an inner mirror at a suitable position, by obtaining the position of the face of a driver, on the basis of images shot from different positions by moving a shooting device; and a method of adjusting the angle thereof. <P>SOLUTION: The inner mirror device comprises an inner mirror enabling the driver to check a rear part of a vehicle, the shooting device arranged in a predetermined positional relationship with the inner mirror and shooting the driver, an inner mirror rotating mechanism rotatably supporting the inner mirror and the shooting device about at least two rotating shafts, a detecting device detecting a position of the driver on the basis of image output of the driver shot from different positions on each rotating shaft of the inner mirror rotating mechanism by the shooting device, and a rotation drive controlling device controlling the rotation drive of the inner mirror and the shooting device on the basis of the position of the face of the driver detected by the detecting device. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an inner mirror device mounted on a vehicle such as an automobile, and more particularly to an inner mirror device that is automatically adjusted and an angle adjustment method thereof.

As a conventional inner mirror device, for example, one or a plurality of cameras for photographing a driver, detection means for detecting a facial feature point and a facial orientation of the driver from an image output of the camera, and extracted facial feature points Based on the orientation of the face, the position of the face when the driver is facing the front is obtained, the means for obtaining the optimum angle of the inner mirror based on the obtained face position, and the optimum angle for which the inner mirror is obtained There is an inner mirror device provided with a means for adjusting (Patent Document 1). According to this invention, the inner mirror adjustment device determines the position of the driver's face, and the inner mirror is adjusted to the optimum angle according to the position of the face, so the driver can adjust the angle of the inner mirror by himself. It can be lost.
JP 2002-274265 A

  However, according to the inner mirror device of Patent Document 1, a facial feature point and a face orientation of a driver are extracted from an image output of a fixed camera, and a face is extracted from the extracted facial feature points and facial orientation. The driver's face position is determined based on the model database. For this reason, when obtaining the position of the driver's face, an error is likely to occur, and the angle adjustment accuracy of the inner mirror is not sufficient.

  An object of the present invention is to provide an inner mirror capable of automatically and accurately adjusting the angle of the inner mirror to an appropriate position by obtaining the position of the driver's face based on images taken from different positions by moving the photographing device. An apparatus and an angle adjustment method thereof are provided.

  According to an aspect of the present invention, an inner mirror that allows a driver to check the rear of a vehicle, a photographing device that is disposed in a predetermined positional relationship with the inner mirror, and photographs the driver, the inner mirror, and the photographing device are provided. An inner mirror rotating mechanism that is rotatably supported around at least two rotating shafts, and an image output of the driver taken by the photographing device from different positions on the rotating shafts of the rotating shafts of the inner mirror rotating mechanism A detection device that detects the position of the driver's face based on the rotation of the driver, and a rotation drive control device that controls the rotation of the inner mirror and the imaging device based on the position of the driver's face detected by the detection device. An inner mirror device is provided.

  According to another aspect of the present invention, an initial setting step for initially setting an angle of an inner mirror for a driver to check the rear of a vehicle and a photographing device for photographing the driver, and the driver photographed by the photographing device. A first image processing calculation step for image processing to calculate a face position in an initial state of the driver; an inner mirror rotation driving step for rotating the inner mirror by a predetermined angle; and the inner mirror A second image processing calculation step of performing image processing on the image of the driver imaged by the imaging device after the rotation driving step, and calculating a position of the face after the inner mirror rotation driving step of the driver; The driver calculated in the first image processing calculation step and the second image processing calculation step Based on the position of the face, providing the angle adjusting method for a rearview mirror device having a rotation drive control step of controlling the rotation angle of the inner mirror.

  According to the embodiment of the present invention, the angle of the inner mirror is automatically and accurately adjusted to an appropriate position by obtaining the position of the driver's face based on images taken from different positions by moving the photographing device. It is possible to provide an inner mirror device and an angle adjustment method thereof.

(Embodiment of the present invention)
FIG. 1 is a diagram showing a vehicle 1 such as an automobile equipped with an inner mirror device according to an embodiment of the present invention, (a) is a plan view seen from above, and (b) is a side view. FIG. The vehicle width direction of the vehicle 1 is the X-axis, the front-rear direction is the Y-axis, the up-down direction is the Z-axis, and the inner mirror rotation mechanism 200 of the inner mirror device 100 is about Φx and the Z axis around the X axis. Initially set in a state rotated by Φz and mounted on the vehicle 1.

  FIG. 2 is a functional configuration diagram of the inner mirror device 100 according to the embodiment of the present invention. The inner mirror device 100 is an inner mirror rotation mechanism 200 for the driver DR to confirm the rear of the vehicle 1, a rotation drive device 300 that rotationally drives the inner mirror rotation mechanism 200, and one photographing device that photographs the driver. The camera 400 includes an image processing device 500 that detects the position of the driver's face by image processing, and an inner mirror and a rotation control device 600 that calculates and controls the rotation control angle of the camera. The ECU 1 is controlled by an ECU (Electrical Control Unit) 700 as needed.

  FIG. 3 is a perspective view showing the configuration of the inner mirror rotating mechanism 200 used in the inner mirror device 100 according to the embodiment of the present invention. The inner mirror 201 is formed on the surface of the inner mirror frame 202, and is supported so as to be rotatable about the x axis and the z axis around the origin O located on the inner mirror surface. The inner mirror frame 202 is supported by the outer frame 203 so as to be rotatable around the x axis, and the outer frame 203 is rotatably supported at a predetermined position of the vehicle 1 via a fixing post 204 while being rotatable around the z axis. It is supported.

  The inner mirror frame 202 is rotationally driven around the x axis while being supported by the x axis rotation support shaft 205 by the x axis rotation actuator 301, and is also z axis rotated by the z axis rotation actuator 302 via the outer frame 203. It is configured to rotate around. Note that the inner mirror frame 202 and the outer frame 203 are preferably rotatably supported by rotation axes orthogonal to each other.

The inner mirror frame 202 has θx, θz with respect to the x axis and the z axis at positions on the inner mirror 201 on the surface thereof at a predetermined distance a _x , a _{z in the} x and z directions from the origin O, respectively. One camera 400 is attached with an angle of λ and an infrared light irradiation device 401 for projecting and shooting invisible light is attached. Note that the infrared light irradiation device 401 may be attached not to the inner mirror frame 202 but to the outer frame 203 or other places such as the vehicle 1, and at a position where the subject at the time of shooting with the camera 400 can be irradiated. I just need it.

(Angle adjustment method of inner mirror device 100 according to an embodiment of the present invention)
FIG. 4 is a process flow diagram illustrating a method for adjusting the inner mirror device 100. Hereinafter, the angle adjustment method of the inner mirror device 100 will be described according to the steps shown in the flowchart.

  FIG. 5A shows that the inner mirror 201 is initially set in a positional relationship in which the driver DR looks directly behind the vehicle 1 when the driver DR with an average seat height sits in the driver seat at the average seat position of the vehicle 1. FIG. 6 is a top plan view showing the positional relationship and angle between the inner mirror rotating mechanism 200 of the inner mirror device 100 and the driver DR, and FIG. It is a figure which shows the positional relationship on a screen.

  Steps s1 to s5 describe an automatic adjustment method of the inner mirror 201 for the driver DR to perform correct backward confirmation in the xy plane (around the z axis), that is, in the horizontal direction.

[Step s1] Initial setting step (horizontal direction)
First, in the horizontal direction, when the average driver DR sits down, the camera 400 mounted on the inner mirror device 100 can be seen directly behind the vehicle 1 and the driver DR is displayed in the approximate center of the camera image. Initialize the angle. The angle of view of the camera 400 is set to reflect the entire range of the driver seat and the assumed seat height range. In this initial state, the angle between the camera 400 and the inner mirror 201 is θz, and the distance from the origin O that is the center of the camera 400 and the inner mirror 201 is a _x . Further, the angle with the X axis of the inner mirror 201 in this state is φz, and the z rotation actuator 302 is rotated by φz from the ECU 700 via the rotation control device 600 when the vehicle engine is started or stopped. Perform initial setting around the axis.

[Step s2] First image processing calculation step (horizontal direction)
For an arbitrary driver DR having a different driver seat position, seat height, and the like, the position of the face of the driver DR is obtained from the image captured by the camera 400 by performing image processing using the image processing apparatus 500.

  FIG. 6 is a diagram for explaining the light projection difference detection method used in the embodiment of the present invention. FIG. 6A is based on a video image projected to the driver DR and a video image not performed. An outline for creating a difference image will be described. FIG. 5B illustrates an outline for creating a difference image particularly when near-infrared light is projected onto the driver DR.

  In the projected image, the effect of applying light to a portion close to the light source appears strongly, and the influence on the background portion 800 far from the light source hardly appears. Therefore, if the difference between the projected image and the non-projected image is taken, it is close to the light source. Only the driver DR which is a part is extracted. A portion extracted by the difference is set as a rough face position of the driver DR. If the difference is extracted including other than the face, only the face part is extracted by template matching using the standard pattern of the face image, and the center coordinates of the face are detected as the face position.

When near-infrared light is projected onto the driver DR, the reflected light from the pupil of the driver DR is particularly strong, and only the left eye close to the camera 400 can be extracted. Let the coordinates be the position of the face of the driver DR. The x coordinate of the face position obtained as described above is defined as b _x .

  Next, the angle of the face position in the initial state of the driver DR obtained by the face position detection is obtained.

FIG. 7A is a diagram showing the angle of the driver DR based on the face position of the driver DR and the angle of view of the camera 400, FIG. 7B is a diagram showing the relationship between the angles in the horizontal direction, and FIG. It is a figure which shows the relationship of the angle of a perpendicular direction. The horizontal screen width W _{h at} the y-direction position of the driver DR in the initial state corresponding to the angle of view of the camera 400 and the distance from the center position calculated from the face position coordinates of the driver DR to the face position b _x From the relationship, the angle of the face position of the driver DR is obtained as α _h × b _x / W _h , which is α _z .

FIG. 8 is a diagram illustrating the direction of the face position of the driver DR in the initial state and the direction of the face position of the driver DR when the inner mirror 201 is rotated by γ _z around the z axis.

Thus, the center of the inner mirror 201 is the origin O, and the position of the driver DR is
y = tan (90 ° −α _z + θz) × (x + a _x ).

[Step s3] Inner mirror rotation drive step (horizontal direction)
Next, in order to obtain the angle of the face position of the driver DR with the inner mirror 201 rotated by γ _z , the inner mirror 201 is rotated around the z axis by γ _{z in} order to obtain the angle of the face of the driver DR.

[Step s4] Second image processing calculation step (horizontal direction)
By the same image processing as in step s2, the x coordinate of the face position of the driver DR is obtained, and this is set as c _x .

Next, the angle of the face position of the driver DR obtained by detecting the face position in a state where the inner mirror 201 is rotated by γ _{z is} obtained as α _h × c _x / W _h , which is expressed as β _z To do.

Next, an angle ζ _z between the center of the inner mirror 201 and the driver DR is calculated. From the angle beta _z position of the face of the driver DR, the position of the driver DR will be in the straight line _{y = tan (90 ° -β z} + θz + ζ z) × (x + a x · cosζ z) on ₊ a x · _{sinζ z} .

[Step s5] Rotation drive control step (horizontal direction automatic adjustment)
Next, the face position of the driver DR is detected, and based on this, the angle of the inner mirror 201 is corrected.

From the intersection of the two straight lines obtained in steps s2 and s4, the position (xd, yd) of the driver DR in the coordinate system having the origin O as the center of the inner mirror 201 is obtained. The angle ζ _z between the center of the inner mirror 201 and the driver DR at the angle of the inner mirror 201 at the initial setting is ζ _z = tan ⁻¹ (yd / xd).

FIG. 9 shows the angular relationship between the inner mirror 201 and the driver DR, and shows the angular relationship that allows the driver DR to see directly behind the vehicle 1 in the horizontal direction, that is, to correctly confirm the rear. In order for the driver DR to perform correct backward confirmation, in FIG. 9, (ζ _z −90 °) = Φz may be satisfied. When the angle of the inner mirror 201 is rotated by the z-axis rotary actuator 302 by (ζ _z −Φz−90 °) / 2, ζ _z becomes ζ _z ′, Φz becomes Φz ′, and ζ _z ′ and Φz ′ are respectively
ζ _z ′ = ζ _z − (ζ _z −Φz−90 °) / 2 = (ζ _z + Φz + 90 °) / 2
Φz ′ = Φz + (ζ _z −Φz−90 °) / 2 = (ζ _z + Φz + 90 °) / 2
It becomes. Therefore, ζ _z ′ −90 ° = (ζ _z + Φz−90 °) / 2 = Φz ′ is obtained.

Therefore, based on the angle Φz of the inner mirror 201 in the initial state and the angle ζ _z between the center of the inner mirror 201 and the driver DR, the z-axis rotation actuator 302 moves the inner mirror 201 around the z axis (ζ _z −Φz By rotating by −90 °) / 2, the horizontal adjustment of the inner mirror 201 is performed in the xy plane so that the driver DR performs correct rearward confirmation.

  Next, an automatic adjustment method of the inner mirror 201 for the driver DR to perform correct rearward confirmation in the yz plane (around the x axis), that is, in the vertical plane, from step s6 to step s10 will be described. The description of the same parts as those in the horizontal automatic adjustment method shown in steps s1 to s5 will be omitted.

[Step s6] Initial setting step (vertical direction)
First, in the vertical direction, the angle of the camera 400 mounted on the inner mirror device 100 is initially set so that the driver DR appears in the approximate center of the camera image. The angle of view of the camera 400 is set to reflect the entire range of the driver seat and the assumed seat height range. In this initial state, the angle between the camera 400 and the inner mirror 201 is θx, and the distance from the origin O that is the center of the camera 400 and the inner mirror 201 is _az . In this state, the angle of the inner mirror 201 with respect to the Z axis is φx, and the x-rotation actuator 301 is driven to rotate by φx from the ECU 700 via the rotation control device 600 when the engine is started or stopped. Perform initial setting around the axis.

  The initial setting may be performed simultaneously with the initial setting in the horizontal direction in step s1.

[Step s7] First image processing calculation step (vertical direction)
For an arbitrary driver DR having a different driver seat position, seat height, and the like, the position of the face of the driver DR is obtained from the image captured by the camera 400 by performing image processing using the image processing apparatus 500. Similarly to step s2, by taking the difference between the difference between the projected image and the non-projected image, the z coordinate of the face position of the driver DR is obtained, and this is _defined as b _z .

Next, the angle of the face position in the initial state of the driver DR obtained by the face position detection is obtained. From FIGS. 7 (a) and (c), the center position calculated and the vertical screen width W _p in the y-direction position of the driver DR in the initial state corresponding to the angle of the camera 400, the position coordinates of the face of the driver DR The angle of the face position of the driver DR is determined as α _p × b _z / W _p from the relationship with the distance from the face position b _z to the face position b _z , which is α _x .

Although not shown, based on the same concept as in FIG. 8, the position of the driver DR when the inner mirror 201 is rotated about the x axis by γ _x on the yz plane is the straight line z = tan (90 ° −α _x + Θx) × (y + a _z ).

[Step s8] Inner mirror rotation drive step (vertical direction)
Next, in a state of rotating the inner mirror 201 only gamma _x, for determining the angular position of the face of the driver DR, gamma _x only rotationally drives the inner mirror 201 about the x-axis by x-axis rotation actuator 301.

[Step s9] Second image processing calculation step (vertical direction)
By the same image processing as in step s4, calculated z coordinates of the position of the face of the driver DR, which is referred to as c _z.

The angle of the face position of the driver DR obtained by detecting the face position in a state where the inner mirror 201 is rotated by γ _{x is} obtained as α _p × c _z / W _p, and this is _defined as β _x .

Next, an angle ζ _x between the center of the inner mirror 201 and the driver DR is calculated. From the angle beta _x position of the face of the driver DR, the position of the driver DR will be in the straight line z = tan on _{(90 ° -β x + θx +} ζ x) × (y + a z · cosζ x) + a z · sinζ x .

[Step s10] Rotation drive control step (vertical direction automatic adjustment)
Next, the position of the face of the driver DR is detected, and based on this, angle correction (vertical direction automatic adjustment) of the inner mirror 201 is performed.

From the intersection of the two straight lines obtained in steps s7 and s9, the position (yd, zd) of the driver DR in the coordinate system with the center of the inner mirror 201 as the origin O is obtained. The angle ζ _x between the center of the inner mirror 201 and the driver DR at the angle of the inner mirror 201 at the initial setting is ζ _x = tan ⁻¹ (zd / yd).

In order for the driver DR to be able to see the vehicle 1 directly behind in the vertical direction, that is, to perform correct rearward confirmation, it is only necessary to satisfy (ζ _x −90 °) = Φx from the same relationship as in FIG. When the angle of the inner mirror 201 is rotated by (ζ _x −Φx−90 °) / 2 by the x-axis rotary actuator 301, ζ _x becomes ζ _x ′, Φx becomes Φx ′, and ζ _x ′ and Φx ′ are respectively
ζ _x ′ = ζ _x − (ζ _x −Φx−90 °) / 2 = (ζ _x + Φx + 90 °) / 2
Φx ′ = Φx + (ζ _x −Φx−90 °) / 2 = (ζ _x + Φx + 90 °) / 2
It becomes. Therefore, ζ _x ′ −90 ° = (ζ _x + Φx−90 °) / 2 = Φx ′ is obtained.

Therefore, the angle of the inner mirror 201 in the initial state [Phi] x, based on the angle zeta _x between the center and the driver DR of the inner mirror 201, (ζ _x -Φx the inner mirror 201 about the x-axis by x-axis rotation actuator 301 By rotating by −90 °) / 2, automatic adjustment in the vertical direction of the inner mirror 201 is performed in the yz plane so that the driver DR performs correct backward confirmation.

(Effect of the embodiment of the present invention)
The embodiment of the present invention has the following effects.
(1) The automatic angle adjustment function using the camera built-in type electric inner mirror and the image processing allows the driver to always obtain an appropriate rear view without bothering the driver.
(2) The driver is photographed by projecting invisible light, and the difference between the projected image and the non-projected image is image-processed by the projected difference detection method to extract the face position or the eye position of the driver DR. Therefore, the position of the driver DR can be easily detected without performing complicated processing.
(3) In this embodiment, a single camera for photographing the driver is used, and the three-dimensional position of the driver can be detected by rotating the camera by a predetermined rotation angle. A practical position can be detected with a simple configuration.
(4) Since the driver is reflected within the camera angle of view, the driver's state can be always properly monitored.
(5) Because the driver's position is known in the inner mirror angle calculation process, it can be applied to various functions depending on the driver's position, such as adjusting the angle of the side mirror, automatically adjusting the air blowing position of the air conditioner, and automatically adjusting the angle of the navigation screen. Is possible.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows vehicles 1, such as a motor vehicle equipped with the inner mirror apparatus which concerns on embodiment of this invention, (a) is the top view seen from upper direction, (b) is the figure seen from the side. is there. It is a functional lineblock diagram of inner mirror device 100 concerning an embodiment of the invention. It is a perspective view which shows the structure of the inner mirror rotation mechanism 200 used with the inner mirror apparatus 100 which concerns on embodiment of this invention. FIG. 6 is a process flow diagram illustrating a method for adjusting the inner mirror device 100. (A) shows that the inner mirror 201 is initially set in a positional relationship in which the driver DR looks directly behind the vehicle 1 when the driver DR with an average seat height sits in the driver seat at the average seat position of the vehicle 1. FIG. 6 is a top plan view of the inner mirror device 100 showing the positional relationship and angle relationship between the inner mirror rotating mechanism 200 of the inner mirror device 100 and the driver DR, and FIG. It is a figure which shows the positional relationship in. It is a figure for demonstrating the light projection difference detection method used by embodiment of this invention, (a) produces a difference image from the image | video which did and did not project light with respect to driver DR. (B) illustrates an outline for creating a difference video particularly when near-infrared light is projected onto the driver DR. (A) is a figure which shows the angle of driver DR from the position of the face of driver DR, and the angle of view of camera 400, (b) is a figure which shows the relationship of the angle of a horizontal direction, (c) is perpendicular | vertical. It is a figure which shows the relationship of the angle of a direction. FIG. 6 is a diagram illustrating a face position direction of a driver DR in an initial state and a face position direction of the driver DR when the inner mirror 201 is rotated by γ _z around the z axis. It is a figure which shows the angular relationship between the inner mirror 201 and driver DR, and can confirm back correctly from driver DR.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle 100 Inner mirror apparatus 200 Inner mirror rotation mechanism 201 Inner mirror 202 Inner mirror frame 203 Outer frame 204 Fixing post 205 x-axis rotation support shaft 300 Rotation drive device 301 x-axis rotation actuator 302 z-axis rotation actuator 400 Camera 401 Infrared Light irradiation device 500 Image processing device 600 Rotation control device 700 ECU
800 Background 801 Left eye DR Driver

Claims (10)

An inner mirror for the driver to check the back of the vehicle,
A photographing device that photographs the driver arranged in a predetermined positional relationship with the inner mirror;
An inner mirror rotation mechanism that rotatably supports the inner mirror and the photographing device around at least two rotation axes;
A detection device for detecting a position of the driver's face based on an image output of the driver imaged by the imaging device from a different position on each rotation axis of the rotation axis of the inner mirror rotation mechanism;
A rotational drive control device that rotationally controls the inner mirror and the imaging device based on the position of the driver's face detected by the detection device;
An inner mirror device.   The inner mirror device according to claim 1, wherein the photographing device photographs invisible light.   The inner mirror device according to claim 1, wherein the detection device detects a position of the face of the driver based on an eye position.   The inner mirror device according to claim 1, wherein the detection device detects a position of the face of the driver using a light projection difference detection method.   The inner mirror device according to claim 1, wherein the detection device detects a position of the driver's face using a template matching method. An initial setting step for initially setting the angle of the inner mirror for the driver to check the rear of the vehicle and the photographing device for shooting the driver;
A first image processing calculation step of performing image processing on the image of the driver photographed by the photographing device and calculating a position of the face in an initial state of the driver;
An inner mirror rotation driving step for rotating the inner mirror by a predetermined angle;
A second image processing calculation step of performing image processing on the image of the driver photographed by the photographing device after the inner mirror rotation driving step and calculating a position of the face after the inner mirror rotation driving step of the driver; ,
A rotational drive control step for rotationally controlling the angle of the inner mirror based on the face position of the driver calculated in the first image processing calculation step and the second image processing calculation step;
The angle adjustment method of the inner mirror apparatus which has this.   The angle adjustment method for an inner mirror device according to claim 6, wherein each of the steps is performed in a horizontal direction and a vertical direction, respectively.   The angle adjustment of the inner mirror device according to claim 6, wherein the first image processing calculation step and the second image processing calculation step detect the position of the face with reference to an eye position. Method.   The inner image according to claim 6, wherein the first image processing calculation step and the second image processing calculation step are performed by performing image processing on the image of the driver using a light projection difference detection method. Angle adjustment method for mirror device. The angle adjustment method for an inner mirror device according to claim 6, wherein the first image processing calculation step and the second image processing calculation step are performed by performing image processing using a template matching method.

Images