JP2007249352A - Information processing device and method, recording medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To quickly and reliably detect the orientation of a driver's face facing forward during driving. <P>SOLUTION: A face detection part 12 detects the position of a driver's face in an input image that is an image captured by a camera 11 to image the driver, who sits in a driver's seat. An initial orientation detection part 13 reads out from a storage part 15 a conversion table associating in advance driver face positions in the input image and initial orientations that are driver face orientations facing forward during driving, and detects an initial orientation according to the conversion table and the driver face position detected by the face detection part 11. The mechanism is applicable to a face orientation detection system for detecting the orientation of a driver's face. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an information processing apparatus and method, a recording medium, and a program, and in particular, information processing that can quickly and surely determine the orientation of a face when a driver is facing the front during driving. The present invention relates to an apparatus and method, a recording medium, and a program.

  Conventionally, it has been proposed to obtain an optical flow between captured images by image processing, and detect the orientation of the driver's face based on the obtained optical flow (see, for example, Patent Document 1).

  By the way, in a driving support device that supports driving that has been actively developed in recent years, for example, when detecting a driver's side look, the direction of the face when the driver is facing the front during driving (hereinafter referred to as the initial direction). Is detected by detecting the direction of the driver's face relative to the driver's face. Therefore, in this case, it is important to set the initial direction as accurately as possible.

  However, in the invention described in Patent Document 1, it is assumed that a camera that captures an image of the driver is installed below the front of the driver, and there is no mention of obtaining an initial direction. Accordingly, since the initial direction cannot be adjusted in accordance with differences in the height of the driver, driving position (driving position), etc., an error may occur between the assumed initial direction and the actual initial direction. Also, as shown in FIG. 1 of Patent Document 1, when the camera is installed below the front of the driver, the steering wheel, the driver's hand, etc. may be in the way, and the driver's face may not be photographed.

  Therefore, conventionally, the number of samples or sampling time for measuring eye position data is set based on the traveling speed of the vehicle and the inter-vehicle distance with the preceding vehicle, and the average value of the measured values at the set number of samples or sampling time is It has been proposed to set the eye position (initial position) when the driver is facing the front (see, for example, Patent Document 2).

  In addition, it has been proposed to set an initial direction (reference direction) by pressing a predetermined switch while the driver maintains a driving posture and faces far away (see, for example, Patent Document 3).

JP 2004-334786 A JP 2004-130940 A Japanese Patent No. 2822508

  However, in the invention described in Patent Document 2, when there is no preceding vehicle, the inter-vehicle distance from the preceding vehicle cannot be detected, so the initial position cannot be set. Further, in the invention described in Patent Document 1, it is necessary to measure the eye position data every time a predetermined number of samples or more than the sampling time, and it takes time to set the initial position.

  Further, in the invention described in Patent Document 3, the driver needs to perform a predetermined operation in order to set the initial direction, the initial direction setting is forgotten, or the head is facing away from the front during actual driving. There is a possibility that the initial direction is set in a state different from the state.

  The present invention has been made in view of such a situation, and makes it possible to quickly and reliably obtain the initial direction.

  An information processing apparatus according to an aspect of the present invention includes a position detection unit that detects a detection position that is a position of a driver's face in an input image that is an image of a driver sitting in a driver's seat, and a driver's face in the input image. Initial direction acquisition that acquires the initial direction corresponding to the detected position from the conversion data that is the data in which the initial position that is the orientation of the face when the driver is facing the front during driving is pre-associated Means.

  In the information processing apparatus according to one aspect of the present invention, a detection position that is a position of the driver's face in the input image that is an image obtained by photographing the driver sitting in the driver's seat is detected, and the position of the driver's face in the input image is detected. The initial direction corresponding to the detected position is acquired from the conversion data, which is data associated in advance with the initial direction, which is the face direction when the driver is facing the front during driving.

  Therefore, based on the position of the driver's face in the input image, the face orientation when the driver is facing the front during driving can be obtained. In addition, the face orientation when the driver is facing the front during driving can be quickly and reliably obtained.

  The position detection means and the initial direction acquisition means are constituted by, for example, a CPU (Central Processing Unit).

  The information processing apparatus may further include conversion means for converting the orientation of the driver's face detected based on the input image into an orientation based on the initial direction.

  Thereby, it is possible to detect the face direction of the driver based on the face direction when the driver is facing the front during driving.

  This conversion means is constituted by, for example, a CPU.

  The information processing apparatus may further include a face direction detecting unit that detects a face direction of the driver based on an input image.

  Thereby, both the direction of the driver's face and the initial direction can be detected.

  This face orientation detection means is constituted by a CPU, for example. Further, the face direction detecting means and the face detecting means can be configured by a dedicated device for detecting the face direction, for example.

  This position detection means detects detection positions for a plurality of input images obtained by photographing a driver sitting in the driver's seat from a plurality of directions, and the initial direction acquisition means uses conversion data corresponding to each input image, The initial direction corresponding to the detection position in each input image can be acquired.

  Thereby, the range which can detect an initial direction can be expanded.

  An information processing method, a program, or a program recorded on a recording medium according to one aspect of the present invention includes a detection position that is a position of a driver's face in an input image that is an image of a driver sitting in a driver's seat. The position detection step to detect, the position of the driver's face in the input image, and the conversion data, which is data in which the initial direction, which is the orientation of the face when the driver faces the front while driving, is associated in advance. And an initial direction acquisition control step for controlling acquisition of an initial direction corresponding to the detected position.

  In an information processing method, program, or program recorded on a recording medium according to one aspect of the present invention, a detection position that is a position of a driver's face in an input image that is an image of a driver sitting in a driver's seat Detected position from the conversion data, which is data in which the position of the driver's face in the input image and the initial direction, which is the face direction when the driver is facing the front during driving, are associated in advance The acquisition of the initial direction corresponding to is controlled.

  Therefore, based on the position of the driver's face in the input image, the face orientation when the driver is facing the front during driving can be obtained. In addition, the face orientation when the driver is facing the front during driving can be quickly and reliably obtained.

  This position detection step includes, for example, a position detection step in which the CPU detects a detection position that is the position of the driver's face in the input image that is an image of the driver sitting in the driver's seat. This initial direction control step Is, for example, a detection position from conversion data that is data in which the position of the driver's face in the input image and the initial direction, which is the orientation of the face when the driver is facing the front during driving, are associated in advance. The initial direction acquisition control step in which the acquisition of the initial direction corresponding to is controlled by the CPU.

  As described above, according to one aspect of the present invention, the face orientation when the driver is facing the front during driving can be obtained based on the position of the driver's face in the input image. In addition, according to one aspect of the present invention, it is possible to quickly and surely determine the orientation of the face when the driver is facing the front during driving.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a first embodiment of a face direction detecting device to which the present invention is applied. The face direction detection device 1 is configured to include a camera 11, a face detection unit 12, an initial direction acquisition unit 13, a face direction detection unit 14, and a storage unit 15.

  The camera 11 captures an image (hereinafter referred to as an input image) obtained by capturing the face of the driver 2 sitting in the driver's seat of the vehicle in which the face direction detection device 1 is provided in the face detection unit 12 and the face direction detection unit 14. Supply.

  The camera 11 is desirably installed at a position where the face of the driver 2 during driving can be photographed as much as possible regardless of differences in the physique and posture of the driver. For example, the camera 11 is installed in the vicinity of the center of the center mirror (room mirror) in the vehicle (where the center mirror is attached to the vehicle), and images the driver 2 from the upper left diagonal direction.

  The face detection unit 12 detects the position of the face of the driver 2 from the input image based on a predetermined method, and supplies information indicating the detection result to the initial direction acquisition unit 13 and the face direction detection unit 14. Note that the method by which the face detection unit 12 detects the position of the face of the driver 2 from the input image is not limited to a specific method, and a method that can detect the position of the face more quickly and accurately is desirable. .

  As will be described later with reference to FIG. 2, the initial direction acquisition unit 13 stores the position of the driver's face in the input image and the face when the driver is facing the front during driving, as stored in the storage unit 15. The initial direction corresponding to the position of the face of the driver 2 in the input image detected by the face detection unit 12 is acquired from a conversion table that is associated with the initial direction that is the direction of The initial direction acquisition unit 13 supplies information indicating the acquired initial direction to the subsequent device of the face direction detection device 1. The conversion table will be described later with reference to FIGS. In the following, the initial direction is represented by the orientation of the face viewed from the camera 11, that is, the orientation of the face in the input image.

  The face orientation detection unit 14 detects the orientation of the face of the driver 2 in the input image as viewed from the camera 11 using a predetermined method. The face orientation detection unit 14 supplies information indicating the detection result to a device subsequent to the face orientation detection device 1. Note that the method in which the face direction detection unit 14 detects the face direction of the driver 2 is not limited to a specific method, and a method that can more quickly and accurately detect the face direction is desirable. For example, details of the method for detecting the face orientation are disclosed in Japanese Patent No. 2982204.

  The subsequent device of the face direction detection device 1 is, for example, based on the information from the initial direction acquisition unit 13 and the detection result from the face direction detection unit 14, the face direction of the driver 2 with respect to the initial direction, That is, the orientation of the face based on the initial direction of the driver 2 is detected, and predetermined processing (for example, detection of the driver's side aside, control of the vehicle, etc.) is performed based on the detection result.

  Next, the face direction detection process executed by the face direction detection apparatus 1 will be described with reference to the flowchart of FIG. This process is started, for example, when supply to the face direction detecting device 1 from a power source for an accessory (ACC) of a vehicle provided with the face direction detecting device 1 is started.

  In step S1, the camera 11 starts photographing around the driver's seat. That is, the camera 11 starts photographing the face of the driver 2 sitting in the driver's seat.

  In step S <b> 2, the face detection unit 12 and the face direction detection unit 14 obtain an input image captured by the camera 11.

  In step S <b> 3, the face detection unit 12 detects the face position of the driver 2. Specifically, first, the face detection unit 12 detects a face region in which the face of the driver 2 is captured from the input image using a predetermined method. FIG. 3 shows an example of an input image, and a face area 31 in which the face of the driver 2 is shown is detected. Next, the face detection unit 12 detects the center point of the detected face area as the face position of the driver 2. In the example of FIG. 3, the center point 32 of the face area 31 is detected as the position of the driver's face.

  In step S4, the face detection unit 12 determines whether or not the face of the driver 2 has been detected in the process of step S3. If it is determined that the face of the driver 2 has been detected, the process proceeds to step S5.

  In step S5, the initial direction acquisition unit 13 acquires the initial direction. Specifically, when it is determined that the face of the driver 2 can be detected, the face detection unit 12 supplies information indicating the detection result to the initial direction acquisition unit 13. The initial direction acquisition unit 13 reads the conversion table stored in the storage unit 15. Here, the conversion table will be described with reference to FIGS.

  FIG. 4 is a view of the driver 2 and the camera 11 as viewed from above, and shows the positional relationship between the driver 2 and the camera 11. In the figure, an arrow 41 indicates the front direction of the vehicle, and positions 42 and 43 indicate the position of the head of the driver 2. In the example of FIG. 4, it is assumed that the camera 11 is installed at the base of the center mirror (room mirror) in the vehicle, that is, at a position where the driver 2 is photographed from the front upper left oblique direction.

  In general, the position of the face when the driver is facing the front during driving depends on the driver or the difference in driving position (for example, the position of the front and back of the seat, the difference in driving posture due to the angle of the backrest, etc.) It changes in the front and rear and up and down directions of the vehicle. On the other hand, since the driver's seat generally does not move in the left-right direction, even if the driver or driving position is different, the position of the face when the driver faces the front during driving hardly changes in the left-right direction. That is, it is considered that the position of the face when the driver is facing the front during driving changes on a two-dimensional plane in the front-rear and vertical directions of the vehicle due to the difference in the driver or driving position.

  In general, even if the driver or the driving position is different, the face direction with respect to the vehicle when the driver is facing the front during driving is substantially the same. In the example of FIG. Direction. Therefore, even if the driver or driving position is different, if the driver's face position is the same, the driver's face orientation viewed from the camera 11 when the driver is facing the front during driving, that is, the initial direction is It can be said that the initial direction is almost different if the driver's face position is almost the same. That is, it can be said that the position of the driver's face relative to the camera 11 and the initial direction substantially correspond to each other.

  For example, when the driver 2 is facing the direction of the arrow 41, when the head of the driver 2 is at the position 42, an angle indicating the face direction of the driver 2 in the left-right direction (horizontal direction) viewed from the camera 11 is θ1. When the head of the driver 2 is at the position 43, θ1 <θ2 when the angle indicating the face direction of the driver 2 in the left-right direction (horizontal direction) viewed from the camera 11 is θ2. In other words, when the driver 2 is facing the front during driving, the tilt (angle) in the left direction increases as the face position of the driver 2 is forward, for example, the driving position of the driver 2 is forward. As the driver's 2 face position is rearward, for example, the driver's 2 driving position is rearward, the leftward inclination (angle) becomes smaller.

  Although not shown, similarly, when the driver 2 is facing the front during driving, the face direction of the driver 2 viewed from the camera 11 is such that the lower the face position of the driver 2 is, for example, The lower the seat height of the driver 2 is, the greater the downward inclination (angle) is. The higher the position of the face of the driver 2 is, for example, the lower the driver 2 is, the smaller the downward inclination (angle) is. Become.

  As described above, the position of the driver's face relative to the camera 11 during driving moves on a two-dimensional plane in the front-rear and vertical directions of the vehicle, and the position of the driver's face relative to the camera 11 and the initial direction are Since there is a one-to-one correspondence, the driver's face position in the input image and the initial direction correspond to a one-to-one correspondence. Therefore, if the position of the driver's face in the input image is known, the initial direction can be estimated almost accurately.

  FIG. 5 is a diagram schematically showing the relationship between the face orientation when the driver is facing the front during driving and the position of the driver's face in the input image. In FIG. 5, the input image is divided into a total of 12 areas of 3 × 4 in the vertical direction. For example, when the position of the driver's face is in the lower left corner area, the driver's face direction in the input image (the driver's face direction as viewed from the camera 11, that is, the initial direction) The face position is substantially the same as that in the upper left corner area, and the vertical direction is substantially the same as in the case where the driver face position is in the lower right corner area. Similarly, when the position of the driver's face is in the upper right corner area, the driver's face orientation in the input image is almost the same as that in the lower right corner area in the left-right direction. The vertical direction is substantially the same as the case where the face position of the driver is in the upper left corner area of the input image.

  FIG. 6 is a diagram illustrating an example of a conversion table. The conversion table is based on the property that the face position and the initial direction of the driver in the input image have a one-to-one correspondence regardless of the difference in the driver or driving position. This data is associated with a direction. In FIG. 6, as in the case of FIG. 5, the input image is divided into a total of 12 regions of 3 × 4 in length, and one initial direction is associated with each region. For example, in the conversion table of FIG. 6, the initial direction corresponding to the upper left corner region of the input image is 45 ° leftward and 0 ° downward, and the initial direction corresponding to the lower right corner region of the input image is The leftward direction is 30 ° and the downward direction is 10 °.

  In addition, by performing measurement or the like in advance for each vehicle type and each assumed installation position of the camera 11, a conversion table corresponding to each vehicle type and installation position is individually created. A conversion table corresponding to the vehicle type and installation position where the camera 11 is installed is used.

  Returning to FIG. 2, in step S5, the initial direction acquisition unit 13 acquires an initial direction corresponding to the face position of the driver detected by the face detection unit 12 from the conversion table.

  In step S6, the initial direction acquisition unit 13 outputs the initial direction. Specifically, the initial direction acquisition unit 13 outputs information indicating the initial direction to a device subsequent to the face orientation detection device 1.

  In step S <b> 7, the face orientation detection unit 14 detects the face orientation of the driver 2. Specifically, when it is determined that the face of the driver 2 can be detected, the face detection unit 12 supplies information indicating the detection result to the face direction detection unit 14. The face orientation detection unit 14 detects the orientation of the driver's face viewed from the camera 11 based on the input image and the face position of the driver 2 detected by the face detection unit 12 using a predetermined method. .

  In step S8, the face direction detection unit 14 determines whether or not the face direction of the driver 2 has been detected in the process of step S7. If it is determined that the face orientation of the driver 2 has been detected, the process proceeds to step S9.

  In step S <b> 9, the face orientation detection unit 14 outputs the face orientation of the driver 2. Specifically, the face direction detection unit 14 outputs information indicating the detection result of the detected face direction of the driver 2 to a device subsequent to the face direction detection device 1. Thereafter, the process proceeds to step S11.

  If it is determined in step S8 that the face direction of the driver 2 cannot be detected, the process proceeds to step S10.

  If the face detection unit 12 determines in step S4 that the face of the driver 2 could not be detected, the process proceeds to step S10.

  In step S10, the face orientation detection unit 14 outputs a detection error. Specifically, when the face of the driver 2 cannot be detected, the face detection unit 12 supplies information indicating that the face of the driver 2 cannot be detected to the initial direction acquisition unit 13 and the face direction detection unit 14. To do. The face direction detection unit 14 outputs information notifying a detection error indicating that the detection of the face direction of the driver 2 has failed to a device subsequent to the face direction detection device 1.

  In step S11, the face direction detection device 1 determines whether or not to end the process. If it is determined not to end the process, the process returns to step S2, and the processes in steps S2 to S11 are repeatedly executed until it is determined in step S11 that the process ends.

  In step S11, for example, when the supply of power for the accessory (ACC) of the vehicle is stopped, the face orientation detection device 1 determines that the process is finished, and the face orientation detection process is finished.

  As described above, even if the driver or the driving position is different, the initial direction corresponding to the driver and the driving position can be obtained almost accurately. In addition, in the state where the driver is sitting on the driver's seat, the initial direction can be promptly obtained by simply detecting the position of the driver's face before starting driving. Furthermore, if only the initial direction is required, it is not necessary to recognize the driver or recognize each organ of the driver's face, so the processing can be lightened and the resolution of the input image can be reduced. Can be suppressed. Even if the image quality of the input image is a little bad, the initial direction can be obtained. Furthermore, the initial direction can be obtained automatically and reliably without the driver performing a special action for obtaining the initial direction.

  Note that the accuracy in the initial direction can be improved by dividing the conversion table into smaller areas.

  Next, a second embodiment of the present invention will be described with reference to FIGS. FIG. 7 is a block diagram illustrating an example of a functional configuration of the face direction detection apparatus 101 according to the second embodiment of the present invention. As will be described later, the face orientation detection device 101 outputs the face orientation of the driver 2 with respect to the initial direction to a subsequent device. The face direction detection device 101 is configured to include a camera 11, a face detection unit 12, an initial direction acquisition unit 13, a face direction detection unit 14, a storage unit 15, and a conversion unit 111.

  In the figure, portions corresponding to those in FIG. 1 are denoted by the same reference numerals, and description of portions having the same processing will be omitted because it will be repeated.

  Compared with the face direction detection apparatus 1 of FIG. 1, the face direction detection apparatus 101 matches in that it includes a camera 11, a face detection unit 12, an initial direction acquisition unit 13, a face direction detection unit 14, and a storage unit 15. The difference is that the converter 111 is included.

  The conversion unit 111 acquires information indicating the initial direction from the initial direction acquisition unit 13 and acquires information indicating the detection result of the face direction of the driver 2 from the face direction detection unit 14. The conversion unit 111 converts the face direction of the driver 2 as viewed from the camera 11 detected by the face direction detection unit 14 into a direction based on the initial direction. The conversion unit 111 outputs information indicating the converted face orientation to a device subsequent to the face orientation detection device 101.

  Next, the face direction detection process executed by the face direction detection apparatus 101 will be described with reference to the flowchart of FIG.

  The processing of steps S101 to S105 is the same as the processing of steps S1 to S5 in FIG.

  In step S106, the face orientation of the driver 2 viewed from the camera 11 is detected in the same manner as in step S7 of FIG.

  In step S107, it is determined whether the face orientation of the driver 2 has been detected, as in the process of step S8 of FIG. If it is determined that the face direction of the driver 2 has been detected, the process proceeds to step S108.

  In step S108, the conversion unit 111 converts the orientation of the detected face. Specifically, the initial direction acquisition unit 13 supplies information indicating the initial direction to the conversion unit 111. The face direction detection unit 14 supplies information indicating the detection result of the face direction of the driver 2 to the conversion unit 111. The conversion unit 111 converts the face direction of the driver 2 as viewed from the camera 11 detected by the face direction detection unit 14 into a direction based on the initial direction.

  Here, a conversion method by the conversion unit 111 will be described with reference to FIG. Here, only the face direction in the left-right direction is considered for the sake of simplicity. The direction of the face 131 indicates the direction of the face of the driver as viewed from the camera 11 when the driver 2 is facing the front, that is, the initial direction, and the direction of the face 132 is detected by the face direction detection unit 14 2 indicates the orientation of the face of the driver 2 with respect to the initial direction, that is, the orientation (angle) obtained by subtracting the orientation of the face 131 from the orientation of the face 132, that is, the orientation of the face 133. It becomes. For example, when the initial direction is 30 ° in the left direction and the face direction of the driver 2 detected by the face direction detection unit 14 is 30 ° in the right direction, the face direction of the driver 2 with respect to the initial direction is It is 60 ° to the right (= 30 ° to the right−30 ° to the left).

  Note that the face direction of the driver 2 with respect to the initial direction is determined in the same way in the vertical direction.

  In step S109, the conversion unit 111 outputs the face orientation of the driver 2. Specifically, the conversion unit 111 outputs information indicating the face direction of the driver with respect to the initial direction to a device subsequent to the face direction detection device 101. Thereafter, the process proceeds to step S110.

  If it is determined in step S104 that the face of the driver 2 cannot be detected, or if it is determined in step S107 that the face direction of the driver 2 cannot be detected, the process proceeds to step S110.

  In step S110, the conversion unit 111 outputs a detection error. Specifically, when the face of the driver 2 cannot be detected, the face detection unit 12 supplies information indicating that the face of the driver 2 cannot be detected to the initial direction acquisition unit 13 and the face direction detection unit 14. To do. Further, when the face direction of the driver 2 cannot be detected, the face direction detection unit 14 supplies information indicating that the face direction of the driver 2 cannot be detected to the conversion unit 111. The conversion unit 111 outputs information indicating that the face orientation of the driver 2 has not been detected to a device subsequent to the face orientation detection device 101.

  In step S111, it is determined whether or not to end the process, similar to the process in step S11 of FIG. If it is determined not to end the process, the process returns to step S102, and the processes in steps S102 to S111 are repeatedly executed until it is determined in step S111 that the process is to be ended.

  If it is determined in step S111 that the process is to be terminated, the face orientation detection process is terminated.

  In this way, the face orientation of the driver 2 with respect to the initial direction is output to the subsequent device of the face orientation detection device 101.

  Note that the initial direction may also be output to a device subsequent to the face orientation detection device 101.

  Next, a third embodiment of the present invention will be described with reference to FIGS. FIG. 10 is a block diagram illustrating an example of a functional configuration of the face direction detection apparatus 201 according to the third embodiment of the present invention. The face orientation detection device 201 is characterized by detecting the face orientation of the driver 2 using a plurality of cameras. The face direction detection device 201 is configured to include cameras 11, 211, a face detection unit 12, an initial direction acquisition unit 13, a face direction detection unit 14, a storage unit 15, and a conversion unit 111.

  In the figure, parts corresponding to those in FIG.

  Compared with the face direction detection device 101 of FIG. 7, the face direction detection device 201 includes a camera 11, a face detection unit 12, an initial direction acquisition unit 13, a face direction detection unit 14, a storage unit 15, and a conversion unit 111. It differs in the point which includes the camera 211 in point.

  The camera 211 is installed at a position different from the camera 11 and is an image (input) of the face of the driver 2 sitting in the driver's seat of the vehicle in which the face orientation detection device 201 is provided from a direction different from the camera 11. Image) is supplied to the face detection unit 12 and the face direction detection unit 14. Hereinafter, when it is necessary to distinguish between an image captured by the camera 11 and an image captured by the camera 211, the image captured by the camera 11 is referred to as a first input image, and the image captured by the camera 211. Is referred to as a second input image.

  FIG. 11 shows an example of installation positions of the camera 11 and the camera 211. The camera 11 is installed, for example, in the vicinity of the base of the center mirror in the vehicle, and images the face of the driver 2 from the upper left diagonal direction. The camera 211 is installed near the upper right end of the dashboard (instrument panel) in front of the driver's seat, and photographs the face of the driver 2 from the diagonally downward front right direction. Thereby, the range in which the face of the driver 2 can be photographed is expanded, and the range in which the direction of the driver's face can be detected is expanded. For example, even when the driver 2 turns his face 90 ° to the camera 11 and the camera 11 cannot capture the face of the driver 2, the camera 211 can capture the direction of the face of the driver 2. The orientation of the second face can be detected.

  The storage unit 15 stores a conversion table corresponding to the first input image, that is, the position of the face of the driver 2 in the first input image and the initial direction viewed from the camera 11 (hereinafter referred to as the first initial direction). A conversion table (hereinafter also referred to as a first conversion table) and a conversion table corresponding to the second input image, that is, the position of the face of the driver 2 in the second input image, A conversion table (hereinafter also referred to as a second conversion table) in association with an initial direction viewed from the camera 211 (hereinafter also referred to as a second initial direction) is stored. The initial direction acquisition unit 13 is based on the position of the face of the driver 2 in the first input image and the first conversion table, or the position of the face of the driver 2 in the second input image and the second conversion table, Detect the initial direction.

  Next, face direction detection processing executed by the face direction detection apparatus 201 will be described with reference to the flowchart of FIG. This process is started, for example, when supply to the face direction detecting device 201 from a power source for an accessory (ACC) of a vehicle provided with the face direction detecting device 201 is started.

  In step S201, the cameras 11 and 211 start photographing around the driver's seat. That is, photographing of the face of the driver 2 sitting in the driver's seat is started by the cameras 11 and 211.

  In step S202, the face detection unit 12 and the face direction detection unit 14 acquire input images (first and second input images) captured by the cameras 11 and 211.

  In step S203, the face detection unit 12 detects the face position of the driver 2 in the same manner as the processing in step S3 in FIG. However, unlike the processing in step S3 in FIG. 2, the face detection unit 12 detects the position of the face of the driver 2 in the first and second input images.

  In step S204, the face detection unit 12 determines whether or not the face position of the driver 2 has been detected. In step S203, the face detection unit 12 determines that the face position of the driver 2 has been detected when the face position of the driver 2 in at least one of the first input image and the second input image can be detected. The process proceeds to step S205.

  In step S205, the initial direction is acquired in the same manner as in step S5 of FIG. However, unlike the processing in step S5, when the face position of the driver 2 is detected in both the first and second input images, the conversion table corresponding to each input image is used to determine the position of each input image. The initial direction corresponding to the face position of the driver 2 is acquired. Further, when the face position of the driver 2 is detected in only one of the first or second input images, the input image is obtained from the conversion table corresponding to the input image in which the face position of the driver 2 is detected. The initial direction corresponding to the face position of the driver 2 is acquired.

  In step S206, the face orientation of the driver 2 is detected in the same manner as in step S7 of FIG. However, unlike the processing in step S7, when the face position of the driver 2 is detected in both the first and second input images, the face direction of the driver 2 is detected based on each input image. Is done. Further, when the face position of the driver 2 is detected in only one of the first or second input images, the face direction of the driver 2 is determined based on the input image from which the face position of the driver 2 is detected. Is detected.

  In step S207, the face orientation detection unit 14 determines whether the face orientation of the driver 2 has been detected. In step S206, the face direction detection unit 14 detects the face direction of the driver 2 when the face direction of the driver 2 can be detected based on at least one of the first or second input images. The process proceeds to step S208.

  In step S208, the conversion unit 111 converts the orientation of the detected face. Specifically, the initial direction acquisition unit 13 supplies information indicating the detection result of the initial direction to the conversion unit 111. The face direction detection unit 14 supplies information indicating the detection result of the face direction of the driver 2 to the conversion unit 111. The conversion unit 111 converts the face direction of the driver 2 detected by the face direction detection unit 14 into a direction based on the initial direction, similarly to the process of step S108 in FIG. 8 described above.

  However, when both the face direction of the driver 2 based on the first input image and the face direction of the driver 2 based on the second input image are detected, the face based on one of the input images Is selected, and the orientation of the selected face is converted into an orientation based on the initial direction. For example, the face direction closer to the front as viewed from the camera 11 or the camera 211 is selected.

  In step S209, as in the process of step S109 of FIG. 8 described above, information indicating the driver's face orientation with respect to the initial direction is output to a device subsequent to the face orientation detection device 201. Thereafter, the process proceeds to step S211.

  If it is determined in step S204 that the face of the driver 2 cannot be detected, or if it is determined in step S207 that the face direction of the driver 2 cannot be detected, the process proceeds to step S210.

  In step S210, a detection error is output in the same manner as in step S110 of FIG.

  In step S211, it is determined whether or not to end the process, similar to the process in step S11 of FIG. If it is determined not to end the process, the process returns to step S202, and the processes in steps S202 to S211 are repeatedly executed until it is determined in step S211 that the process is to be ended.

  If it is determined in step S211 to end the process, the face orientation detection process ends.

  Thus, the detection range of the face direction of the driver 2 can be expanded.

  In the above description, an example in which the number of cameras is two has been described, but three or more cameras may be provided. In this case, as in the case where the number of cameras is two, conversion tables corresponding to input images taken by the respective cameras are used.

  In addition, the initial direction may be output to the subsequent device of the face direction detection device 201, or the driver as viewed from the initial direction and the camera 11 or the camera 211 as in the face direction detection device 1. The orientation of the second face may be output.

  Next, a fourth embodiment of the present invention will be described with reference to FIGS. FIG. 13 is a block diagram illustrating an example of a functional configuration of the face direction detection device 301 according to the fourth embodiment of the present invention. The face orientation detection device 301 is characterized by appropriately updating the conversion table by registering or deleting the initial direction by learning processing without using the conversion table in which the initial direction is registered in advance. The face direction detection device 301 includes a camera 11, a face detection unit 12, a face direction detection unit 14, a storage unit 15, a conversion unit 111, an initial direction acquisition unit 311, a front direction determination unit 312, an initial direction detection unit 313, and a conversion. A table update unit 314 is included.

  In the figure, portions corresponding to those in FIG. 7 are denoted by the same reference numerals, and the description of portions having the same processing will be omitted because it will be repeated.

  Compared with the face direction detection device 101 of FIG. 7, the face direction detection device 301 is identical in that it includes the camera 11, the face detection unit 12, the face direction detection unit 14, the storage unit 15, and the conversion unit 111. The difference is that it includes a direction acquisition unit 311, a front direction determination unit 312, an initial direction detection unit 313, and a conversion table update unit 314, but does not include the initial direction acquisition unit 13.

  Similar to the initial direction acquisition unit 13 shown in FIG. 7, the initial direction acquisition unit 311 corresponds to the initial direction corresponding to the face position of the driver 2 detected by the face detection unit 12 from the conversion table stored in the storage unit 15. To get. The initial direction acquisition unit 311 supplies information indicating the acquired initial direction to the conversion unit 111.

  In addition, the initial direction acquisition unit 311 displays information indicating that the initial direction is not registered when the initial direction corresponding to the face position of the driver 2 detected by the face detection unit 12 is not registered in the conversion table. And supplied to the front direction determination unit 312 and the initial direction detection unit 313.

  The front direction determination unit 312 acquires information indicating the detection result of the face direction of the driver 2 from the face direction detection unit 14. Further, the front direction determination unit 312 acquires information indicating various states of the vehicle from a vehicle control unit (not shown) that controls various operations of the vehicle in which the face direction detection device 301 is provided. As described later with reference to FIG. 22, the front direction determination unit 312 is based on the detection result of the face direction of the driver 2 and the state of the vehicle, and the driver 2 faces the front (the front direction of the vehicle). Information indicating the determination result is supplied to the initial direction detection unit 313.

  The initial direction detection unit 313 acquires information indicating the detection result of the face position of the driver 2 from the face detection unit 12. Further, the initial direction detection unit 313 acquires information indicating the detection result of the face direction of the driver 2 from the face direction detection unit 14. When the initial direction corresponding to the face position of the driver 2 detected by the face detection unit 12 is not registered in the conversion table stored in the storage unit 15, the initial direction detection unit 313 refers to FIG. As will be described later, the initial direction is detected by taking statistics of the orientation of the face of the driver 2 when the driver 2 determines that the driver 2 is facing the front. The initial direction detection unit 313 supplies information indicating the detection result to the conversion table update unit 314.

  The conversion table update unit 314 is detected by the face position of the driver 2 in the input image detected by the face detection unit 12 and the initial direction detection unit 313 as described later with reference to FIGS. 16 to 21. The conversion table is updated based on the initial direction. Further, the conversion table update unit 314 supplies information indicating the update result of the conversion table to the initial direction acquisition unit 311.

  Next, face direction detection processing executed by the face direction detection device 301 will be described with reference to the flowchart of FIG.

  Note that the processing of steps S301 to S304 is the same as the processing of steps S101 to S104 of FIG. 8 described above, and the description thereof will be omitted because it will be repeated.

  In step S305, the face orientation of the driver 2 is detected in the same manner as in step S106 in FIG.

  In step S306, similarly to the process of step S107 of FIG. 8 described above, it is determined whether or not the face direction of the driver 2 can be detected. If it is determined that the face direction of the driver 2 can be detected, the process is as follows. The process proceeds to step S307.

  In step S307, the initial direction acquisition unit 311 determines whether or not the initial direction with respect to the face position of the driver 2 is registered. Specifically, the initial direction acquisition unit 311 reads the conversion table stored in the storage unit 15.

  FIG. 15 is a diagram illustrating an example of a conversion table. In the conversion table of FIG. 15, the area where the leftward and downward angles are shown is an area where the initial direction has already been registered. For example, the initial direction corresponding to the upper right corner region is 30 ° leftward and 0 ° downward. Also, the area indicated by diagonal lines is an area where the initial direction has not yet been learned and the initial direction is not registered.

  When the initial direction corresponding to the driver's face position detected by the face detection unit 12 is not registered in the conversion table, the initial direction acquisition unit 311, for example, in the example of FIG. If the position is within the hatched area, it is determined that the initial direction for the face position of the driver 2 is not registered, and the process proceeds to step S308.

  In step S308, the face direction detection device 301 executes a conversion table learning process. The details of the conversion table learning process will be described later with reference to FIG. 16, but the conversion table is learned by this process, and the conversion table is updated as necessary.

  In step S309, as in the process of step S307, it is determined whether the initial direction for the face position of the driver 2 is registered. If it is determined that the initial direction for the face position of the driver 2 is registered, that is, the initial direction for the detected face position of the driver 2 is newly registered in the conversion table by the conversion table learning process in step S308. If so, the process proceeds to step S310.

  On the other hand, when the initial direction corresponding to the position of the driver's face detected by the face detection unit 12 is registered in the conversion table in S307, for example, the detected position of the face of the driver 2 in the example of FIG. Is in an area other than the area indicated by hatching, it is determined that the initial direction for the face position of the driver 2 is registered, and the processing of steps S308 to S309 is skipped, that is, the conversion table learning processing is performed. Instead, the process proceeds to step S310.

  In step S310, the initial direction acquisition unit 311 acquires the initial direction corresponding to the position of the driver's face detected by the face detection unit 12 from the conversion table, as in the process of step S105 of FIG.

  In step S311, the face orientation of the detected driver 2 is converted to a reference orientation with respect to the initial direction, as in the processing of step S108 in FIG. 8 described above.

  In step S312, information indicating the driver's face orientation relative to the initial direction is output to a device subsequent to the face orientation detection device 301, as in the processing of step S109 of FIG. Thereafter, the process proceeds to step S315.

  If it is determined in step S309 that the initial direction corresponding to the face position of the driver 2 is not registered, that is, the initial direction with respect to the detected face position of the driver 2 is determined by the conversion table learning process in step S308. If not registered in the conversion table, the process proceeds to step S313.

  In step S313, the conversion unit 111 outputs initial direction unregistered. Specifically, the initial direction acquisition unit 311 supplies the conversion unit 111 with information indicating that the initial direction is not registered. The conversion unit 111 outputs information notifying the initial direction unregistration indicating that the face direction of the driver 2 has not been detected because the initial direction is not registered, to a device subsequent to the face direction detection device 301. Thereafter, the process proceeds to step S315.

  If it is determined in step S304 that the face of the driver 2 cannot be detected, or if it is determined in step S306 that the face direction of the driver 2 cannot be detected, the process proceeds to step S314.

  In step S314, a detection error is output in the same manner as in step S110 of FIG. Thereafter, the process proceeds to step S315.

  In step S315, it is determined whether or not the process is to be terminated, similar to the process in step S111 of FIG. 8 described above. If it is determined not to end the process, the process returns to step S302, and the processes in steps S302 to S315 are repeatedly executed until it is determined in step S315 that the process is to be ended.

  If it is determined in step S315 that the process is to be terminated, the face orientation detection process is terminated.

  Next, details of the conversion table learning process in step S308 of FIG. 14 will be described with reference to the flowchart of FIG.

  In step S331, the face orientation detection device 301 executes an initial direction detection process. Details of the initial direction detection process will be described later with reference to FIG. 22, but it is determined by this process whether or not the driver 2 is facing the front, and when it is determined that the driver 2 is facing the front, An initial direction corresponding to the current face position of the driver 2 is detected. Further, information indicating the detection result is supplied from the initial direction detection unit 313 to the conversion table update unit 314.

  In step S332, the conversion table update unit 314 determines whether or not the initial direction has been detected. If it is determined that the initial direction has been detected, the process proceeds to step S333.

  In step S333, the conversion table update unit 314 determines whether or not the registered initial direction and consistency are registered. Specifically, the conversion table update unit 314 reads the conversion table from the storage unit 15. The conversion table update unit 314 checks whether or not the initial direction detected by the initial direction detection unit 313 is consistent with the initial direction already registered in the conversion table.

  Here, the consistency between the detected initial direction and the registered initial direction will be described with reference to FIG. FIG. 17 is a diagram illustrating an example of a conversion table. Note that FIG. 17 shows an example of a conversion table corresponding to a case where the input image is divided into a total of six areas of 2 × 3 in the vertical direction.

  For example, in the conversion table of FIG. 17, the initial directions corresponding to the regions 331-1 and 331-6 have not yet been learned (because they have not been learned), and are not registered, and the regions 331-3 to 331-5 are not registered. Let us consider a case where the initial direction corresponding to the region 331-2 is newly detected when the initial direction corresponding to is already registered.

  Since the region 331-2 is in the same row (same height) as the adjacent region 331-1 and region 331-3, the vertical direction (angle) in the initial direction corresponding to the region 331-2 is the region 331. -1 and the vertical direction of the initial direction corresponding to the region 331-3 are estimated to be close. Further, since the region 331-2 is in the same column (the same position in the left-right direction) as the adjacent region 331-5, the horizontal direction in the initial direction corresponding to the region 331-2 corresponds to the region 331-5. It is estimated that the initial direction is close to the horizontal direction.

  For example, if the initial direction corresponding to the region 331-2 is detected as 35 ° leftward and 20 ° downward, the horizontal direction is the same as the initial direction corresponding to the region 331-5. However, regarding the vertical direction, the difference from the vertical direction in the initial direction corresponding to the region 331-3 is 15 °, which is greatly different. For example, when a predetermined threshold value for determining whether or not consistency is obtained is ± 10 °, the initial direction corresponding to the detected region 331-2 is consistent with the registered initial direction. It is determined that there is no.

  Thus, the conversion table update unit 314 has the initial direction detected by the initial direction acquisition unit 13 (the initial direction to be registered) and the area corresponding to the initial direction to be registered (the face position of the driver 2). ) If the difference from the initial direction corresponding to the region (position) in the vicinity of) is within a predetermined threshold, it is determined that consistency is achieved, and if the difference exceeds the predetermined threshold, the consistency is It is determined that it has not been removed.

  If it is determined in step S333 that the registered initial direction is consistent, the process proceeds to step S334.

  In step S334, the conversion table update unit 314 registers the detected initial direction in the conversion table.

  In step S335, the conversion table update unit 314 performs linear interpolation on the conversion table. Here, the linear interpolation of the conversion table will be described with reference to FIGS. 18 shows an example of a conversion table before linear interpolation, and FIG. 19 shows an example of a conversion table after linear interpolation. 18 and 19 show examples of conversion tables corresponding to a case where the input image is divided into a total of 8 areas of 2 × 4 in the vertical direction.

  In the conversion table of FIG. 18, the initial directions corresponding to the areas 332-1 to 332-3 and 332-6 to 332-8 are not registered because they have not yet been learned, and the initial directions corresponding to the area 332-4 are not registered. Is considered when the initial direction corresponding to the area 332-5 is newly detected.

  As described above, since the position of the face of the driver 2 moves substantially on the secondary plane, it can be assumed that the initial direction changes almost in proportion to the position of the face of the driver 2 in the input image. Assuming that, for the regions 332-1 to 332-3 and 332-6 to 332-8 between the region 332-4 and the region 332-5, initials corresponding to the regions 332-4 and 332-5 Linear interpolation using the direction is possible.

  That is, in the vertical direction and the horizontal direction between the region 332-4 and the region 332-5, it is assumed that the initial direction changes at equal intervals with respect to the change in position. The direction is set to the left direction 45 and the downward direction 5 °, the initial direction corresponding to the region 332-2 is set to the left direction 40 ° and the downward direction 5 °, and the initial direction corresponding to the region 332-3 is set. The initial direction corresponding to the region 332-6 is set to the left direction 40 ° and the downward direction 10 °, and the initial direction corresponding to the region 332-7 is set to the left. The direction is set to 35 ° and the downward direction is set to 10 °, and the initial direction corresponding to the region 332-8 is set to the left direction 30 ° and the downward direction 5 °. That is, it is possible to linearly interpolate the initial direction corresponding to the region (position) where the initial direction is not registered, using the initial direction registered in the conversion table.

  When the initial direction changes according to a predetermined rule that is not equally spaced, for example, when it is known that the initial direction changes according to a predetermined function such as a quadratic function or a sine function, the initial direction is interpolated according to the rule. It is also possible to do so.

  Further, this linear interpolation process is not necessarily performed. That is, only the actually detected initial direction may be registered in the conversion table without performing the linear interpolation process.

  Thereafter, the process proceeds to step S337.

  If it is determined in step S333 that the registered initial direction is not consistent, the process proceeds to step S336.

  In step S336, the conversion table update unit 314 deletes the initial direction. For example, as in the example of FIG. 17, when the initial direction corresponding to the newly detected region 331-2 is not consistent with the initial direction registered in the conversion table, the conversion table update unit 314 As shown in FIG. 20, the detected initial direction is deleted as it is without being registered in the conversion table. That is, the initial direction corresponding to the region 331-2 remains unregistered. In addition, for example, as illustrated in FIG. 21, the conversion table update unit 314 further deletes the initial direction corresponding to the region adjacent to the region 331-2 (position near the region 331-1) from the conversion table. .

  Thereafter, the process proceeds to step S337.

  If it is determined in step S332 that the initial direction has not been detected, the process proceeds to step S337.

  In step S337, the conversion table update unit 314 supplies the update result of the conversion table to the initial direction acquisition unit 311 and the conversion table learning process ends.

  Next, details of the initial direction detection processing in step S331 in FIG. 16 will be described with reference to the flowchart in FIG.

  In step S351, the front direction determination unit 312 determines whether the vehicle is moving forward at a predetermined speed or higher. Specifically, the front direction determination unit 312 receives information indicating the vehicle speed and the gear position of the transmission (power transmission device) of the vehicle from the vehicle control unit of the vehicle in which the face direction detection device 301 is provided. And whether or not the vehicle is moving forward at a predetermined speed or higher is determined based on the acquired information. If it is determined that the vehicle is moving forward at a predetermined speed or more, the process proceeds to step S352.

  In step S352, the front direction determination unit 312 determines whether or not the state where the change in the position and orientation of the face of the driver 2 is within a predetermined range has continued for a predetermined time. Specifically, the front orientation determination unit 312 determines that the face position of the driver 2 is within one area in the conversion table from the frame a predetermined time before the current frame to the current frame, When the amount of change in the orientation is within a predetermined range (for example, within a range of 30 ° in the vertical direction and the horizontal direction), the state in which the change in the position and orientation of the driver 2 is within the predetermined range is predetermined. It is determined that the time has continued, and the process proceeds to step S353. That is, this is because the state where the vehicle moves forward at a predetermined speed or more and the change of the face position and orientation of the driver 2 is within a predetermined range continues for a predetermined time, and the driver 2 faces the front. This is the case.

  In step S353, the initial direction detection unit 313 detects the initial direction, and the initial direction detection process ends. Specifically, the front direction determination unit 312 supplies information indicating that the driver 2 is facing the front to the initial direction detection unit 313. The initial direction detection unit 313 takes statistics of the driver's face orientation at a predetermined time when a change in the position and orientation of the driver 2 is within a predetermined range continues for a predetermined time. Find the initial direction. For example, the initial direction detection unit 313 obtains the average value of the face direction of the driver 2 or the peak value in the distribution of the face direction of the driver 2 as the initial direction at the predetermined time. The initial direction detection unit 313 supplies information indicating the detection result to the conversion table update unit 314.

  If it is determined in step S351 that the vehicle is not moving forward at a predetermined speed or higher, the process proceeds to step S354.

  In step S354, the front direction determination unit 312 resets the measurement value. Specifically, the front direction determination unit 312 resets the measurement value of the duration time when the change in the position and orientation of the face of the driver 2 is within a predetermined range, and starts re-measurement.

  If it is determined in step S352 that the change in the face position and orientation of the driver 2 is within a predetermined range does not continue for a predetermined time, the process proceeds to step S355.

  In step S355, the initial direction detection unit 313 notifies that the initial direction has not been detected. Specifically, the front direction determination unit 312 supplies information indicating that the driver 2 cannot be determined to be facing the front to the initial direction detection unit 313. The initial direction detection unit 313 supplies information indicating that the initial direction has not been detected to the conversion table update unit 314.

  In this manner, regardless of the vehicle type, the installation position of the camera 11, or the positional relationship between the camera 11 and the driver's seat, an optimal conversion table corresponding to the difference can be generated. There is no need to prepare a conversion table corresponding to each installation position in advance. Further, even if the vehicle type or the installation position of the camera 11 is different, it is not necessary to change the face orientation detection device 301, and it is possible to realize common parts and cost reduction. Further, in the same vehicle type, for example, even if the specifications of the position of the steering wheel (so-called right handle or left handle) are different, the face direction detecting device 301 can be used without any change.

  In addition, when the initial direction is not registered in the conversion table, information indicating unregistration is output to the subsequent device, so the subsequent device invalidates the processing using the face direction of the driver 2 and malfunctions. Can be prevented. Further, when the initial direction is learned, the process can be resumed.

  Note that a plurality of cameras may be provided in the face orientation detection device 301 as in the face orientation detection device 201 of FIG. FIG. 23 is a block diagram showing an embodiment of a face direction detection device 401 that detects the face direction of the driver 2 using two cameras by adding a camera 411 to the face direction detection device 301. .

  Compared with the face direction detection device 301 in FIG. 13, the face direction detection device 401 is a camera 11, a face detection unit 12, a face direction detection unit 14, a storage unit 15, a conversion unit 111, an initial direction acquisition unit 311, and a front direction. They are identical in that they include a determination unit 312, an initial direction detection unit 313, and a conversion table update unit 314, and differ in that they include a camera 411.

  Although detailed description of the face direction detection processing by the face direction detection device 401 is omitted, conversion tables corresponding to the input images of both the camera 11 and the camera 411 are learned by the same processing as the face direction detection device 301 described above. Based on the two cameras and the conversion table, the orientation of the face of the driver 2 is detected. Thereby, the detection range of the face direction of the driver 2 can be expanded.

  Of course, it is possible to use three or more cameras.

  In addition to the above-described example, the present invention can be applied to, for example, the face of a person who drives or operates various vehicles (for example, a car driver, a train driver, an aircraft pilot, a ship pilot, etc.). The present invention can be applied to a device that detects the orientation, a device that detects the orientation of the face of the operator of the machine tool, and the like.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a program recording medium in a general-purpose personal computer or the like.

  FIG. 24 is a block diagram showing an example of the configuration of a personal computer 900 that executes the above-described series of processing by a program. A CPU (Central Processing Unit) 901 executes various processes according to a program stored in a ROM (Read Only Memory) 902 or a recording unit 908. A RAM (Random Access Memory) 903 appropriately stores programs executed by the CPU 901, data, and the like. The CPU 901, ROM 902, and RAM 903 are connected to each other by a bus 904.

  An input / output interface 905 is also connected to the CPU 901 via the bus 904. The input / output interface 905 is connected to an input unit 906 made up of a keyboard, mouse, microphone, etc., and an output unit 907 made up of a display, a speaker, etc. The CPU 901 executes various processes in response to a command input from the input unit 906. Then, the CPU 901 outputs the processing result to the output unit 907.

  The recording unit 908 connected to the input / output interface 905 includes, for example, a hard disk, and stores programs executed by the CPU 901 and various data. A communication unit 909 communicates with an external device via a network such as the Internet or a local area network.

  Further, the program may be acquired via the communication unit 909 and stored in the recording unit 908.

  A drive 910 connected to the input / output interface 905 drives a removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives the program or data recorded therein. Get etc. The acquired program and data are transferred to and stored in the recording unit 908 as necessary.

  As shown in FIG. 24, a program recording medium for storing a program that is installed in a computer and is ready to be executed by the computer includes a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only). Memory, DVD (Digital Versatile Disc), a magneto-optical disk, a removable medium 911 which is a package medium composed of a semiconductor memory, a ROM 902 in which a program is temporarily or permanently stored, and a recording unit 908 It is comprised by the hard disk etc. which comprise. The program is stored in the program recording medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 909 that is an interface such as a router or a modem as necessary. Done.

  In the present specification, the step of describing the program stored in the program recording medium is not limited to the processing performed in time series in the order described, but is not necessarily performed in time series. Or the process performed separately is also included.

  Furthermore, the embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention.

It is a block diagram which shows 1st Embodiment of the face direction detection apparatus to which this invention is applied. It is a flowchart for demonstrating the face direction detection process performed by the face direction detection apparatus of FIG. It is a figure for demonstrating the position of the face of a driver in an input image. It is a figure for demonstrating direction of the face of the driver seen from the camera. It is a figure which shows typically the relationship between the direction of the face in the case of facing the driver front, and the position of the face of the driver in an input image. It is a figure which shows the example of a conversion table. It is a block diagram which shows 2nd Embodiment of the face direction detection apparatus to which this invention is applied. It is a flowchart for demonstrating the face direction detection process performed by the face direction detection apparatus of FIG. It is a figure for demonstrating the method to convert the direction of the face of the detected driver into the direction on the basis of an initial direction. It is a block diagram which shows 3rd Embodiment of the face direction detection apparatus to which this invention is applied. It is a figure which shows the example of the installation position of the camera of FIG. It is a flowchart for demonstrating the face direction detection process performed by the face direction detection apparatus of FIG. It is a block diagram which shows 4th Embodiment of the face direction detection apparatus to which this invention is applied. It is a flowchart for demonstrating the face direction detection process performed by the face direction detection apparatus of FIG. It is a figure which shows the example of a conversion table. It is a flowchart for demonstrating the detail of the conversion table learning process of FIG.14 S308. It is a figure for demonstrating the consistency with the detected initial direction and the registered initial direction. It is a figure for demonstrating the linear interpolation of a conversion table. It is a figure for demonstrating the linear interpolation of a conversion table. It is a figure for demonstrating the method to erase | eliminate an initial direction from a conversion table. It is a figure for demonstrating the other method of deleting an initial direction from a conversion table. It is a flowchart for demonstrating the detail of the initial direction detection process of FIG.16 S331. It is a block diagram which shows 5th Embodiment of the face direction detection apparatus to which this invention is applied. And FIG. 11 is a block diagram illustrating an example of a configuration of a personal computer.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Face direction detection apparatus 2 Driver 11 Camera 12 Face detection part 13 Initial direction acquisition part 14 Face direction detection part 15 Memory | storage part 101 Face direction detection apparatus 111 Conversion part 201 Face direction detection apparatus 211 Camera 301 Face direction detection apparatus 311 Initial direction acquisition Unit 312 front direction determination unit 313 initial direction detection unit 314 conversion table update unit 401 face direction detection device 411 camera 901 CPU
902 ROM
903 RAM
908 Recording unit 910 Drive 911 Removable media

Claims (7)

Position detection means for detecting a detection position that is a position of the driver's face in the input image, which is an image obtained by photographing the driver sitting in the driver's seat;
From the conversion data, which is data in which the position of the driver's face in the input image and the initial direction, which is the face direction when the driver is facing the front during driving, is previously associated with the detected position An information processing apparatus comprising: initial direction acquisition means for acquiring the corresponding initial direction. The information processing apparatus according to claim 1, further comprising conversion means for converting a direction of the face of the driver detected based on the input image into a direction based on the initial direction. The information processing apparatus according to claim 1, further comprising a face orientation detection unit that detects a face orientation of the driver based on the input image. The position detection means detects the detection position for a plurality of input images obtained by photographing a driver sitting in the driver's seat from a plurality of directions,
The information processing apparatus according to claim 1, wherein the initial direction acquisition unit acquires the initial direction corresponding to the detection position in each input image from the conversion data corresponding to each input image. A position detection step for detecting a detection position that is a position of the face of the driver in the input image, which is an image obtained by photographing the driver sitting in the driver's seat;
The detected position from the conversion data, which is data in which the position of the driver's face in the input image and the initial direction, which is the face direction when the driver faces the front during driving, are associated in advance. And an initial direction acquisition control step for controlling acquisition of the initial direction corresponding to. A position detection step for detecting a detection position that is a position of the face of the driver in the input image, which is an image obtained by photographing the driver sitting in the driver's seat;
The detected position from the conversion data, which is data in which the position of the driver's face in the input image and the initial direction, which is the face direction when the driver faces the front during driving, are associated in advance. A program that causes a computer to execute information processing including an initial direction acquisition control step that controls acquisition of the initial direction corresponding to the.   A recording medium on which the program according to claim 6 is recorded.

Images