JP2006253787A - Image input device and passenger monitoring device for vehicle provided with the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image input device for acquiring the image information of an object as a recognition target by stabilizing it under prescribed conditions. <P>SOLUTION: The image input device 10 acquires the image information of the object existing near a prescribed position. The device is provided with a photographing means 2 for photographing the object, a displacement detection means 3 for detecting the displacement of the existing position of the object, and a control means 4 for controlling either of or both of a photographing direction and photographing magnification on the basis of the detection result of the displacement means 3. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image input apparatus that acquires image information of an object that is assumed to exist in the vicinity of a predetermined position. The present invention also relates to a vehicle occupant monitoring device that recognizes a face of an occupant seated on a vehicle seat as a predetermined position as an object.

  As an example of an image input device that acquires image information of an object that is assumed to exist in the vicinity of a predetermined position, there is a camera having a shooting mode switching function. For example, when taking a person with a personal camera as a subject and taking a snapshot or the like, it can be assumed that the person who is the target is approximately 2 to 5 m in front of the camera. Also, in the case of a security camera or a personal authentication camera provided in an automatic teller machine or the like, it is possible to assume a person who operates the automatic teller machine, particularly the position of a face portion. Similarly, occupant and occupant face positions are also assumed in occupant monitoring devices that detect changes in driving conditions by monitoring the driver's or passenger's face orientation, gaze movement, etc. it can.

  As described above, in various scenes, image information of an object assumed to exist in the vicinity of a predetermined position is acquired, and the image information is utilized. Even in the above-described personal camera, it is possible to perform color correction, prevention of red eyes, etc. by recognizing a person's face part, and the technique for recognizing a person's face is very useful.

  Therefore, various techniques for recognizing a person's face have been proposed to date. For example, Patent Literature 1 shown below proposes a technique for recognizing a human face mainly in a humanoid robot, an entertainment robot, or the like. The purpose of this is to detect the face with high efficiency even when the face of the person seen from these robots is tilted or in a different direction. In this technique, first, a correlation between a template image having a predetermined size indicating an average face image and an input image is obtained. When the correlation between the average template image and the input image is low, the correlation is obtained using the template image rotated by a predetermined angle with respect to the vertical axis of the screen. Thus, by rotating one template image to obtain a correlation (matching), even if the target is not a front face, matching can be performed using a front face template. As a result, the speed of the matching process is improved.

Japanese Patent Laying-Open No. 2004-133737 (paragraph 9-11)

  With the above technique, the template matching speed can be improved to some extent, but when the recognition target changes other than rotation, this speed improvement effect is not exhibited. For example, when the recognition target moves in the front-rear direction with respect to the photographing means (for example, a camera), there is a difference between the size of the template image and the size of the recognition target. Due to the nature of template matching, a high correlation value cannot be obtained if there is a deviation in size. As a method for dealing with this, template images corresponding to a plurality of sizes can be prepared, but the processing time is increased. If matching with a template image corresponding to a plurality of sizes is performed in parallel in order not to increase the processing time, the scale of the apparatus (the scale of the electronic circuit, the number of arithmetic units, etc.) increases.

  This invention is made | formed in view of such a subject, and it aims at providing the image input device for acquiring stably the image information of the target object used as recognition object on a predetermined condition.

In order to achieve the above object, the characteristic configuration of the image input apparatus according to the present invention is as follows:
Obtaining image information of an object existing near a predetermined position,
An imaging means for imaging the object; a displacement detection means for detecting a displacement of an existing position of the object;
Control means for controlling either one or both of the photographing direction and the photographing magnification based on the detection result of the displacement detecting means.

  According to this characteristic configuration, the control unit controls the shooting direction and the shooting magnification based on the result of detecting the displacement of the position where the object exists. That is, as the control of the photographing direction, the photographing direction can be changed (pan, tilt, swing) by the photographing means, and the photographed image can be shifted. Further, as control of the photographing magnification, an optical zoom by changing an optical system such as a lens of a photographing unit, a digital zoom using a photographed image, or the like can be performed. As a result, the image information of the object to be recognized can be stabilized under certain conditions, and image processing using the image information, for example, image recognition can be performed smoothly.

  The displacement detecting means is a distance detecting means for detecting a distance between the object and the photographing means, and the control means controls the photographing magnification based on a detection result of the distance detecting means. Is preferable.

  As described above, when it is assumed that the object exists in the vicinity of the predetermined position, the behavior of the object can be monitored. According to the above configuration, this monitoring is performed by the distance detection means. And since imaging magnification is controlled based on the detection result of a distance detection means, the image information of a target object can be stabilized on fixed conditions.

  Here, it is preferable that the control means determines the photographing magnification so that the object has a predetermined size in the image information.

In this way, since the control unit controls the photographing magnification based on the detection result of the distance detection unit, the image information of the object can be stabilized within a certain condition, that is, within a range of the same size. it can.
In the background art described above, the template image can be changed based on the detection result, but for this purpose, it is necessary to store template images corresponding to a plurality of sizes. Alternatively, image processing for changing the size of the template image based on the detection result is required. These cause an increase in the scale of image processing means (devices, circuits, etc.) and an increase in processing time.
However, if the imaging magnification is controlled as in the above configuration, the image information handled by the image processing means can be stabilized. It is also possible to perform so-called pipeline processing for image processing and control of photographing magnification.

  Furthermore, it is preferable that the image input device as described above is provided in a vehicle occupant monitoring device that recognizes the face of the vehicle occupant seated on the vehicle seat as the object.

  An occupant driving a vehicle is seated in a predetermined driver's seat when driving. Therefore, it is assumed that the occupant as an object exists in the vicinity of the predetermined position. The displacement of the occupant's position, that is, the behavior does not vary greatly during operation. Therefore, by using the displacement detection means as described above, it falls within the category of detectable displacement. Therefore, for example, in order to monitor the driving situation such as the occupant's facial expression, face orientation, etc., it is possible to stably obtain image information of a certain location. As a result, it is possible to obtain an accurate vehicle occupant monitoring apparatus capable of suppressing the scale of the image processing means and performing high-speed processing.

  Embodiments of the present invention will be described below with reference to the drawings, taking a vehicle occupant monitoring device as an example. FIG. 1 is a perspective view illustrating the positional relationship between an occupant seated in a vehicle according to an embodiment of the present invention and a camera that photographs the occupant. As shown in FIG. 1, the occupant is seated in a driver seat in the vehicle and takes a driving posture. The camera 2 is installed in the room mirror 7 in this example, and can photograph the head 1 including the face of the occupant from almost the front. Note that the installation location of the camera 2 is not limited to this as long as the head 1 of the occupant can be photographed from almost the front. The sun visor part provided in the driver's seat or the ceiling part above it may be used. A meter panel unit provided with various meters or a dashboard unit provided with audios may be used.

  In the normal driving posture, the occupant sits on the driver's seat and faces the front to operate the vehicle. Even if the posture is slightly changed by visual confirmation of the surroundings, the head 1 may be considered to be present in the vicinity of the predetermined position. In other words, it may be considered that the headrest 6 is approximately in the vicinity of the headrest 6 of the driver's seat.

  FIG. 2 is a diagram illustrating an example of an image obtained by photographing an occupant from the camera 2 of FIG. In the figure, the range surrounded by the thick frame W indicates the captured image P. As shown in FIG. 2, when an occupant's face is to be recognized using a captured image P showing a relatively wide area, first, “head (1)” needs to be extracted from the captured image P. There is. At this time, the shape of the headrest 6 and the window 8 included in the captured image P may be erroneously determined as the “head”. Therefore, when a wide-angle image such as the photographed image P is used, there are concerns about problems such as an increase in processing time and erroneous detection.

  Therefore, in order to avoid such a problem, the occupant's face is captured almost entirely in the shooting frame. The size of the human head is not significantly different, although there are individual differences. Therefore, the occupant's head 1 that is a subject (object) that has a size within a predetermined range and is present in the vicinity of the predetermined position is stably captured almost completely in the imaging frame.

  FIG. 3 is a diagram illustrating an example of an image obtained by photographing the object (occupant's face) almost entirely in the photographing frame. In the standard captured image P0 shown in FIG. 3, the occupant's head 1 is imaged approximately at the center of the imaging frame. In the case of a system that digitally processes acquired image information as in the present embodiment, a digital camera is generally used as the camera 2. In general, a digital camera is a landscape image whose aspect ratio is 3 to 4. Therefore, even if the occupant's posture is slightly displaced left and right, the displaced occupant's heads 1A and 1B are within the range of the standard captured image P0.

  However, the displacement of the occupant's posture is not limited to the above. There are also displacements in the vertical direction and displacements in the front-rear direction. Therefore, in the present invention, displacement detection means for detecting the displacement of the position of the occupant's head 1 as the object is provided, and based on the detection result of the displacement detection means, either one of the shooting direction and the shooting magnification or Both sides are controlled.

  FIG. 4 is a schematic block diagram showing a system configuration of a vehicle occupant monitoring device using such an image input device. As shown in the figure, the image input unit (image input device) 10 includes a camera 2 (imaging unit), a distance sensor 3 (distance detection unit) as a displacement detection unit, and a control unit 4 (control unit). ing. As the distance sensor 3, a well-known device such as an optical distance sensor using infrared rays or a sonar distance sensor using ultrasonic waves can be used as appropriate. In the present embodiment, for ease of explanation, a case where an optical distance sensor 3 is used as a displacement detection unit will be described as an example. When the distance sensor 3 is used, it is possible to detect the displacement in the front-rear direction of the occupant's head 1 as a subject. If the detection target area is arranged at a wide angle or a plurality of displacement detection means are used, it is possible to detect the displacement in the vertical direction and the horizontal direction in addition to the front-rear direction.

  The camera 2 is controlled by the control unit 4 based on the detection result of the displacement detection means such as the distance sensor 3, and photographs the occupant's head 1 as an object while maintaining a constant size. The camera 2 includes a zoom mechanism 51 and a rotation mechanism 52 as the captured image control means 50 as shown in the figure. The zoom mechanism 51 changes the focal length by changing the optical system of the camera 2 and changes the magnification for photographing the subject (occupant's head 1).

  The rotation mechanism 52 rotates the direction of the optical system (or the entire camera 2) of the camera 2 in each direction such as pan, tilt, and swing. Here, pan, tilt, and swing are used in the same meaning as commonly used terms. The pan rotates the camera 2 left and right around the vertical axis while keeping the camera 2 horizontal. Tilt rotates around the center of the lens. Swing is a vertical rotation about a horizontal axis.

  In this way, the control unit 4 changes one or more of the shooting direction (pan, tilt, swing) and the shooting magnification (zoom) of the camera 2 according to the displacement of the object detected by the displacement detection means. . That is, any one or more of the rotation mechanism 52 and the zoom mechanism 51 are operated so that the occupant's head 1 in the captured image P (image information) has a predetermined size.

  This control will be described below by taking as an example a case where the photographing magnification is changed based on the detection result by the distance sensor 3. FIG. 5 is an explanatory diagram showing the relationship between the occupant's head 1 and the distance sensor 3. The distance sensor 3 is adjusted so that the distance to the object can be detected with high accuracy in the detection range S shown in FIG. Since the object exists in the vicinity of the predetermined position, it is possible to narrow the detection range to some extent and set the detection accuracy high.

  As shown in FIG. 5, when the occupant's head 1 is in the standard position in the detection range S, the distance is detected as the standard distance L0. When the occupant's head 1 is displaced in the front-rear direction, for example, when it becomes the front position object 1C, the distance is detected as the approaching distance L1. In the case of the rear position object 1D, the distance is detected as the separation distance L2.

When the distance to the object is detected in this way, the control unit 4 controls the shooting magnification based on this distance. That is, the photographing magnification is determined so that the object has a predetermined size. FIG. 6 is a diagram for explaining the control of the photographing magnification at each position in FIG. The shooting magnification is determined based on the case where the head 1 of the passenger is present at the standard distance L0.
Shooting magnification = detection distance / standard distance L0 (Equation 1)
Indicated by

As shown in the upper part of FIG. 6, when the occupant's head 1 is ahead of the standard position,
Shooting magnification = approach distance L1 / standard distance L0 (Equation 2)
And
Approaching distance L1 <standard distance L0 (Equation 3)
Therefore, the photographing magnification is smaller than 1. That is, since the head 1 of the occupant comes out forward and appears large in the captured image P, the zoom is reduced. On the contrary, when the head 1 of the occupant is present behind the standard position, the photographing magnification is larger than 1 as shown in the lower part of FIG. This is clear from the fact that the separation distance L2 is substituted in the expression 2 instead of the approach distance L1.

  Thus, stable image information can be obtained by simple linear calculation. In the above calculation example, the description is made on the assumption that the camera 2 and the distance sensor 3 are substantially at the same position for easy explanation. Even when the camera 2 and the distance sensor 3 are not installed at the same position, although it is more complicated than the equations 1 to 3, it can be easily calculated by linear calculation.

For example, as shown in FIG. 7, a case where the camera 2 and the distance sensor 3 are arranged at a predetermined installation distance L3 will be described. In this case, the distance f0 between the object in the standard position (occupant's head 1) and the camera 2 is a value obtained by adding the standard distance L0 and the installation distance L3. Further, the distance f1 between the front position object 1C and the camera 2 is a value obtained by adding the approach distance L1 and the installation distance L3.
When the occupant's head 1 is in front of the standard position as in FIG.
Shooting magnification = (L1 + L3) / (L0 + L3) (Formula 4)
Therefore, from the formula 3, the photographing magnification is smaller than 1. That is, the zoom is reduced. Thus, stable image information can be obtained by simple linear calculation in this case as well.

  Further, the same applies to the case where the camera 2 and the distance sensor 3 are arranged not only in the front-rear direction with respect to the object but also in a position shifted in the horizontal direction or the vertical direction. The arrangement relationship between the camera 2 and the distance sensor 3 is known from design values or measured values at the time of installation. Therefore, the calculation may be similarly performed by well-known linear calculation such as triangulation.

  Image information (photographed images P0, P1, and P2) obtained stably as described above is sent from the image input unit 10 to the image processing unit 20 as shown in FIG. 4, where recognition processing is performed. The recognition result is determined by the determination unit 30, and various determinations (estimation) are made according to the direction of the occupant's face, facial expression, and the like. For example, it is possible to estimate aside driving based on the direction of the occupant's face, or to estimate doze based on the movement of the pupil or eyelid. In accordance with this determination, various post-processing such as issuing a notification / warning to the occupant or controlling the traveling speed of the vehicle can be performed.

  Note that each part in the vehicle occupant monitoring device 40 shown in FIG. 4 indicates the sharing as a function, and does not necessarily indicate a physically independent one. For example, the control unit 4, the image processing unit 20, and the determination unit 30 may be common hardware such as one processor, one semiconductor IC, or a group of electronic circuits. It is sufficient that each function is shared by software such as a program executed on the hardware.

  The same applies to the zoom mechanism 51 and the rotation mechanism 52 in the camera 2. The zoom mechanism 51 is not limited to optical zoom, and may be a method using electronic zoom (digital zoom). The rotation mechanism 52 may also change the cut-out position when a wide-angle image is taken and a predetermined area is cut out from the image. When using the digital signal processing method as described above, the camera 2 does not need to include the captured image control means 50 as shown in FIG. In other words, the captured image correction unit 5 may be provided separately and used as the captured image control means. The captured image correction unit 5 may be provided in the image processing unit 20. Further, the same hardware as that of the control unit 4 and the determination unit 30 may be used in the same manner as described above. These parts do not necessarily have to be physically independent as described above, and may be sufficient if they share each function.

As described above, according to the present invention, it is possible to provide an image input device for stably acquiring image information of an object to be recognized under a predetermined condition. Further, by using this image input device, a vehicle occupant monitoring device for recognizing the occupant's face sitting on the vehicle seat as a predetermined position as an object can be obtained.
Furthermore, the image input device of the present invention is an input device for recognizing a person's face in various automated devices such as a cash dispenser, automatic teller machine, age-limited vending machine (cigarette, liquor, etc.), etc. Available to:

The perspective view explaining the positional relationship of the passenger | crew seated in the vehicle which concerns on embodiment of this invention, and the camera which image | photographs this passenger | crew The figure which shows an example of the image which image | photographed the passenger | crew from the camera of FIG. The figure which shows the example of the image which image | photographed the target object (occupant's face) from the camera of FIG. 1 is a schematic block diagram showing an example of the system configuration of an image input apparatus according to an embodiment of the present invention and a vehicle occupant monitoring apparatus using the same. Explanatory drawing which shows the relationship between the target object (occupant's head) and distance sensor which concern on this invention FIG. 5 is a diagram for explaining control of photographing magnification at each position in FIG. The figure explaining control of photographing magnification when a camera and a distance sensor are arranged apart 1 is a schematic block diagram showing another system configuration example of an image input apparatus according to an embodiment of the present invention and a vehicle occupant monitoring apparatus using the same.

Explanation of symbols

1 head (object)
2 Camera (photographing means)
3 Distance sensor (displacement detection means)
4 Control unit (control means)
10 Image input device 50 Photographed image control means

Claims (4)

An image input device for acquiring image information of an object existing near a predetermined position,
Photographing means for photographing the object;
Displacement detecting means for detecting displacement of the position of the object;
An image input device comprising: control means for controlling either or both of the photographing direction and the photographing magnification based on the detection result of the displacement detecting means.   The displacement detection unit is a distance detection unit that detects a distance between the object and the imaging unit, and the control unit controls the imaging magnification based on a detection result of the distance detection unit. The image input device described in 1.   The image input device according to claim 1, wherein the control unit determines the photographing magnification so that the object has a predetermined size in the image information.   A vehicle occupant monitoring device comprising the image input device according to any one of claims 1 to 3 and recognizing a face of an occupant seated on a vehicle seat as the object.

Images