JP2006279796A - Cellphone, holder therefor, program therefor, and semiconductor device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a cellphone which can measure an automotive driver's physical body with a high precision and can contribute to a safety driving. <P>SOLUTION: This cellphone recognizes a human's face, eyes and mouth and the like from an image obtained by a camera 10. Further, simultaneously, the cellphone measures a time period from an utterance of conversation voices from a speaker 12 to an input of responsive voices to a microphone 11. The cellphone measures a frequency of a blink, the number of breathing and a conversation responsive time and the like per unit time to create reference data. When a measurement value goes out of the reference data at a given ratio or higher, it is considered that an abnormality occurs in a driver's physical body condition to issue an alarm. The alarm may be issued from the speaker 12, or issued from, for instance, an automotive audio apparatus or the like by communication with the other apparatus. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a mobile phone, a mobile phone holder, a mobile phone program, and a semiconductor device for grasping the physical condition of a car driver.

  In recent years, various methods have been considered to improve traffic safety. For example, since snoozing due to fatigue of a driver of a car is a major cause of an accident, in recent years, a device that recognizes the physical condition of the driver and issues a warning has been proposed (for example, see Patent Document 1).

This device monitors a driver's driving situation with a CCD camera, and detects the degree of fatigue by detecting temporal changes in the driver's white eye area by image processing. It is also possible to measure the degree of tension from the position where the handle is gripped and the current voltage between both hands.
Japanese Patent Laid-Open No. 11-175895

  However, the driver situation-dependent communication device described in Patent Document 1 detects the degree of fatigue only from the time change of the driver's white eye area. Thus, just judging the driver's fatigue level with a single item increases the number of errors, and an erroneous alarm may hinder driving.

  Further, the driver situation-dependent communication device described in Patent Document 1 requires that a CCD camera, an electrode for measuring the current / voltage between both hands, etc. be installed in the automobile body, and such a detector is later incorporated into the automobile. Had a very time-consuming problem.

  An object of the present invention is to provide a mobile phone that can measure the body condition of a car driver with high accuracy without having to be incorporated into the car later, and contributes to safe driving.

  According to the first aspect of the present invention, there is provided an image acquisition unit that acquires an image of a human face, an image analysis unit that detects the human body state from the image acquired by the image acquisition unit, and the detected body state is predetermined. And an abnormality notification means for determining that an abnormality occurs when the standard is not met and issuing a warning or outputting information indicating that a warning is issued to the outside.

  In the present invention, an image of a human face is acquired using an image sensor such as a CCD camera. Using the acquired image, image analysis is performed to detect the human body condition. If it is determined that the physical condition is abnormal, a warning is issued. For example, a warning voice is emitted from a speaker of a mobile phone. In addition, a warning sound may be emitted from the car audio device using a separate communication means such as BlueTooth. Further, a warning may be output to the outside. Using the call function of the mobile phone, the center server is notified of the occurrence of an abnormality. The center server may receive the notification of the occurrence of the abnormality and call the mobile phone, or may transfer the notification to a contact point designated in advance.

  The invention according to claim 2 is characterized in that, in the above-mentioned invention, the image analysis means detects a body state by detecting the number of blinks per a predetermined time or the number of breaths.

  In the present invention, the number of blinks per predetermined time or the number of breaths per predetermined time is detected. For example, the respiratory rate is detected by detecting the heat of breathing in the infrared region of a CCD camera. When the number of blinks or breathing per unit time is out of the standard, it is judged as abnormal.

  According to a third aspect of the present invention, in the above invention, a voice output unit that emits a conversational voice, a voice input unit that inputs a human voice, and a voice input unit that generates a human voice to the voice input unit after the voice output unit emits a conversation voice. It is further characterized by further comprising a conversation reaction detecting means for detecting a physical state by measuring a time until voice is input.

  In the present invention, a conversation voice is emitted, and a time until a person who hears the voice speaks and inputs a response voice is measured. It is determined that the response is abnormal when the time until the input is out of the standard.

  According to a fourth aspect of the present invention, in the above invention, the apparatus further comprises: an acceleration sensor that detects acceleration in three-axis directions; and a driving state determination unit that determines a driving state of the automobile vehicle from the acceleration detected by the acceleration sensor. The abnormality notifying means determines that the abnormality is detected when the driving situation detected by the driving situation judging means is out of a predetermined standard.

  In the present invention, the driving situation of the driver is determined from the attitude of the automobile vehicle detected using the three-axis acceleration sensor. A mobile phone is mounted on a car, and the change in posture of the mobile phone is measured as the change in posture of the car. When the posture variation is out of the standard, it is determined to be abnormal. For example, it is determined that the vehicle is abnormal if the vehicle is running flat.

  According to a fifth aspect of the present invention, in the above invention, a human voice is input after the number of blinks per predetermined time detected by the image analysis means, the respiratory rate, and the conversation voice detected by the conversation reaction detection means are emitted. And at least two items of the driving situation detected by the driving situation determination unit, and the abnormality notification unit issues a warning when it is determined that an abnormality has occurred in any one item It is characterized by that.

  The invention according to claim 6 controls a pedestal for fixing a mobile phone, a data communication unit for performing data communication with the mobile phone, a drive unit for rotating the pedestal, and a rotation angle of the drive unit. A mobile phone holder including a control unit, wherein the control unit controls the drive unit based on a drive control signal input from the mobile phone via the data communication unit. .

  In the present invention, the rotation angle is controlled by receiving a drive control signal from the mobile phone. The drive control signal is information indicating the moving direction and moving amount of the pedestal, and is calculated so that, for example, a specific image (for example, a human face) recognized by the mobile phone comes to the center position of the camera shooting range. . The control unit receives the drive control signal and rotates the pedestal so that the specific image comes to the center position of the camera photographing range. What is necessary is just to rotate a base by 2 axis | shafts of the left-right direction and an up-down direction. As a result, a specific image can be automatically tracked, and a target image can be accurately captured without manual adjustment.

  The invention according to claim 7 controls a pedestal for fixing a mobile phone, a data communication unit for performing data communication with the mobile phone, a drive unit for rotating the pedestal, and a rotation angle of the drive unit. A mobile phone holder including a control unit, wherein the control unit rotates the specific image received from the mobile phone through the data communication unit so that the specific image is displayed at the center of the mobile phone screen. The moving angle is controlled.

  In the present invention, image data is acquired from a mobile phone, and a specific image (for example, a human face) is recognized. A camera direction shift is detected based on the face position in the image, and the pedestal is rotated so that the human face is displayed at the center of the camera shooting range. What is necessary is just to rotate a base by 2 axis | shafts of the left-right direction and an up-down direction. Accordingly, the mobile phone can automatically track a specific image only by transmitting image data, and can accurately capture a target image without manual adjustment.

  The invention according to claim 8 is an image acquisition procedure for acquiring an image of a human face on a mobile phone, a body state detection procedure for detecting the human body state from the image acquired by the image acquisition procedure, and a detected body An abnormality notification procedure for issuing a warning when the state is out of a predetermined standard or outputting information indicating that a warning is issued to the outside is executed.

  According to a ninth aspect of the present invention, in the above invention, the body state detection procedure is a time from when the number of blinks per predetermined time or the number of breaths is detected or when a human voice is input after a conversation voice is generated. It includes a procedure for detecting a physical state by measuring.

  According to a tenth aspect of the present invention, in the above invention, an acceleration detection procedure for detecting triaxial acceleration, a body state detected by the body state detection procedure, and driving of an automobile from the acceleration detected by the acceleration detection procedure An operation condition determination procedure for determining a situation is further executed, and the abnormality notification procedure includes a procedure for determining that an abnormality is detected when the operation condition detected in the operation condition determination procedure is outside a predetermined standard. It is characterized by.

  The invention according to claim 11 is incorporated in a mobile phone, and an image acquisition procedure for acquiring an image of a human face in the mobile phone, and a body for detecting the human body state from the image acquired by the image acquisition procedure. Recorded a mobile phone program for executing a condition detection procedure, an abnormality notification procedure for issuing a warning when the detected physical condition falls outside a predetermined standard, or outputting information indicating that a warning is issued to the outside It is characterized by that.

  According to a twelfth aspect of the present invention, in the above invention, the physical condition detection procedure is a time from detection of the number of blinks per predetermined time or breathing rate or generation of a conversational voice to input of human voice. It includes a procedure for detecting a physical state by measuring.

  According to a thirteenth aspect of the present invention, in the above-described invention, an acceleration detection procedure for detecting acceleration in three axes, a body state detected by the body state detection procedure, and driving of an automobile vehicle from the acceleration detected by the acceleration detection procedure. A program for further executing a driving situation determination procedure for determining a situation is recorded, and the abnormality notification procedure is determined to be abnormal when the driving situation detected in the driving situation determination procedure is out of a predetermined standard. It includes a procedure.

  As described above, according to the present invention, the body state is detected by acquiring an image of a human face with an image sensor such as a CCD camera mounted on a mobile phone or by producing a conversational voice. Without being incorporated, the body condition of the automobile driver can be measured with high accuracy.

  The mobile phone of this embodiment recognizes a human face from an image acquired by an image sensor such as a mounted CCD camera, and detects a body state such as the number of blinks and the number of breaths by analyzing the image of the recognized face. . When it is determined that the body condition is abnormal (blinks, the number of breathing is too small from the reference, etc.), a warning sound is emitted from a speaker or a connected car audio device, which contributes to safe driving.

  Hereinafter, a mobile phone and a mobile phone holder according to embodiments of the present invention will be described with reference to the drawings. FIG. 1 is an external view of a mobile phone. FIG. 2 is an external view of a mobile phone holder.

  As shown in FIG. 1, a mobile phone 1 includes a camera 10 that captures a front image, a microphone 11 that inputs sound, a speaker 12 that outputs incoming sound, a display unit 13 that displays various images, and a plurality of images. The operation unit 14 including buttons is provided. These functional units are connected to a control unit (described later) that controls the entire mobile phone device, and the user of the mobile phone 1 operates the operation unit 14 to perform voice calls, information communication, and the like with the outside. be able to. In addition, still images and moving images can be taken by the camera 10 and stored in a built-in storage unit (not shown), and various image processing can be performed on the acquired images.

  In FIG. 2, the mobile phone holder 2 is fixed to a flat part in an automobile or the like, and is for holding the mobile phone. Moreover, it can operate by receiving power supply from the automobile and can supply power to a battery (not shown) of the mobile phone 1 to be connected. The mobile phone holder 2 holds the mobile phone by being joined to a fixed base 26 fixed to a flat part in an automobile or the like, a circular base 27 connected to the upper part of the fixed base and rotatable, and the circular base 27. An L-shaped table 28 is provided.

  The L-shaped base 28 has an I / O connector (plug) 22 for connecting to an I / O connector (receptacle) 21 (see FIG. 3) at the lower part of the mobile phone on the upper surface. The I / O connector 22 has a convex shape, and is coupled to the cellular phone 1 by fitting the concave I / O connector 21 of the cellular phone 1. The I / O connector 22 includes a contact for supplying power to the mobile phone 1 and a terminal for transmitting / receiving data to / from the mobile phone 1. A sectional view of the L-shaped table 28 is shown in FIG. The cellular phone 1 can be held in the cellular phone holder 2 at the I / O connector 22, the upper surface portion of the L-shaped base 28, and the rear surface portion as shown in FIG. A convex portion may be further provided on the front side of the upper surface of the L-shaped table 28 (front side of the mobile phone 1). Accordingly, the mobile phone can be sandwiched between the back surface and the front surface, and the mobile phone can be held more stably.

  The L-shaped table 28 can be rotated around a horizontal rotation axis by the drive unit 24B. Although not shown, the drive unit 24B includes a motor, a gear mechanism (ohm gear), and the like that generate a driving force for rotating the L-shaped table 28. In this example, the stand for holding the mobile phone 1 is described as an L-type stand, but the external shape is not limited to the L-type as shown, and the mobile phone is held and the horizontal direction is set. Any shape may be used as long as the shaft is rotatable.

  The circular base 27 joined to the L-shaped base 28 can be rotated about a vertical rotation axis by a drive unit 24A provided in the fixed base 26. The drive unit 24A includes a motor, a gear mechanism (ohm gear), and the like that generate a driving force that rotates the circular platform 27, although not shown. In this example, it is described as a circular platform, but the external shape is not limited to a circle, and any shape can be used as long as it is connected to a fixed platform and can rotate about a vertical direction. Also good. Further, the fixed base is not limited to the cylindrical shape as shown in FIG.

  The drive unit 24A and the drive unit 24B are connected to a control unit (described later) built in the mobile phone holder 2 so that the camera 10 of the mobile phone 1 can capture a target image (direction in which an image capturing target exists). The angle of rotation is controlled (towards

  Next, with reference to FIG. 3, a connection configuration of each functional unit of the mobile phone 1 and the mobile phone holder 2 will be described. FIG. 3 is a block diagram showing the configuration of the mobile phone 1 and the mobile phone holder 2. As shown in the figure, a camera 10 as an image acquisition unit, a microphone 11 as an audio input unit, a speaker 12 as an audio output unit, a display unit 13 and an operation unit 14 are connected to a control unit 16 via a bus. It is connected to the. The control unit 16 is connected to the acceleration sensor 15, the RAM 17, the ROM 18, the server communication unit 19, the serial communication unit 20, and the I / O connector 21 via a bus. The sound input to the microphone 11 is converted into sound data by the control unit 16. The voice data is output to the server communication unit 19 and transmitted to another telephone or the like via a center server (not shown). In addition, voice data from another telephone received from the server communication unit 19 is converted into a voice signal by the control unit 16 and is emitted from a speaker for call (not shown). Thereby, a voice conversation with another telephone or the like becomes possible. Note that the speaker 12 for outputting incoming call sound and the like and the speaker for calling may be provided separately, or any one of the speakers may be shared. For example, a call voice may be output from the speaker 12.

  In addition, the control unit 16 may transmit various data such as text data input by the user via the operation unit 14 and image data captured by the camera 10 from the server communication unit 19 to another mobile phone via the center server. Is possible.

  Further, the control unit 16 can perform various processes by reading a program stored in the ROM 18 and developing it in the RAM 17. For example, it is possible to perform image recognition processing for recognizing a specific image for image data captured by the user with the camera 10.

  The acceleration sensor 15 is a three-axis (X-axis, Y-axis, Z-axis) sensor, and detects acceleration in the direction of each axis. The detected acceleration in each axial direction is input to the control unit 16, and the control unit 16 can determine the swinging direction of the mobile phone 1.

  The serial communication unit 20 is an interface that can exchange data with other devices, and is configured by wireless communication such as BlueTooth. Any other serial data communication can be used.

  The I / O connector 21 is a general-purpose interface capable of serial data communication as well as a power supply terminal of the mobile phone 1. Various devices can be connected to the I / O connector 21. In this embodiment, the mobile phone 1 is connected to the I / O connector 22 of the mobile phone holder 2 by connecting the I / O connector 21 to the I / O connector 21. And communication between the mobile phone holder 2.

  The I / O connector 22 of the mobile phone holder 2 is connected to the control unit 23. The control unit 23 performs data transmission / reception with the mobile phone 1 via the I / O connector 22. The control unit 23 is connected to the drive unit 24A and the drive unit 24B, and controls the rotation angle so that the camera 10 of the mobile phone 1 can capture a predetermined image. That is, the following processing is performed.

  When the I / O connector 21 of the mobile phone 1 and the I / O connector 22 of the mobile phone holder 2 are connected, the camera 10 of the mobile phone 1 captures an image and inputs it to the control unit 16. The control unit 16 recognizes a specific image (here, a human face) from the input image. The control unit 16 converts the recognized image into information on the coordinate position of the camera shooting range. Based on this coordinate position, a drive control signal is transmitted to the control unit 23 of the mobile phone holder 2 via the I / O connector 21 and the I / O connector 22. Based on the drive control signal, the control unit 23 drives the drive unit so that the coordinate position of the recognized image matches the center coordinate position of the camera shooting range, that is, the shooting target comes to the center of the shooting range of the camera 10. 24A and the drive unit 24B are controlled. Specifically, as will be described in detail later, the circular base 27 and the L-shaped base 28 are driven by a signal fed back from the image movement direction and the movement deviation due to the change of the direction of the mobile phone 1, and the deviation is eliminated. To do.

  In this manner, the mobile phone holder 2 automatically tracks the human face, and the user can accurately capture a target image without manual adjustment by the user.

  In the above example, the control unit 16 of the mobile phone 1 recognizes a specific image (human face), but the mobile phone 1 may include a processing unit dedicated to image recognition. As the processing unit dedicated to image recognition, for example, a dedicated IC chip or a DSP is used.

  In the above example, the mobile phone recognizes the image to be tracked, determines the coordinates, and transmits the drive control signal to the mobile phone holder 2 so that the shooting target comes to the center of the shooting range. Such image recognition may be performed by the control unit 23 of the mobile phone holder 2, or a processing unit dedicated to image recognition may be provided in the mobile phone holder 2. In this way, the mobile phone 1 can transmit the image captured by the camera 10 only to the mobile phone holder 2, and the mobile phone holder 2 side can perform the automatic tracking process.

  Moreover, although the example in which the automatic tracking process is triggered by the connection of the I / O connector 21 of the mobile phone 1 and the I / O connector 22 of the mobile phone holder 2 has been described, the user operates the mobile phone 1. The unit 14 may be operated to instruct to start the automatic tracking process.

  The image recognition program recognizes an image of a human face based on a predetermined recognition algorithm. For example, an image acquired by the camera 10 is divided into a plurality of small areas, and each image is collated with a face pattern registered in advance. Alternatively, a color image may be acquired to determine whether the color matches the skin color.

  The control unit 23 of the mobile phone holder 2 is connected to the communication unit 25. The communication unit 25 is connected to an audio device of the automobile. The control unit 23 emits an alarm sound for occurrence of abnormality from the audio device of the automobile via the communication unit 25. The communication unit 25 may be configured to be directly connected by wire, or may be connected to an audio device of an automobile by wireless communication such as BlueTooth. In addition, you may make it emit a warning sound from the audio equipment of a motor vehicle via FM broadcast electric wave.

  The occurrence of abnormality here refers to a state in which the control unit 16 of the mobile phone 1 cannot recognize a human face. In addition, the recognized human face is close to the limit of the rotatable range of the circular platform 27 and the L-shaped platform 28 (for example, when the recognized human has moved greatly), and the human 10 is centered on the shooting range of the camera 10. Even when the face cannot be moved, a warning sound is generated as an abnormality. As a warning sound, a sound such as “not recognized” or “out of tracking range” may be emitted.

  Further, a warning sound may be emitted from the speaker 12 of the mobile phone 1. When the human face cannot be recognized, the control unit 16 of the mobile phone 1 emits a warning sound from the speaker 12 using an internal sound source (stored in the ROM 18). Even if the human face is recognized, if the control unit 23 of the mobile phone holder 2 determines that it exists outside the rotatable range of the circular base 27 and the L-shaped base 28, information notifying the occurrence of abnormality is sent to the mobile phone. 1 is transmitted to the control unit 16, and the control unit 16 emits a warning sound from the speaker 12.

  Next, a driver monitor process that is a process for detecting the physical state of the driver will be described. The following driver monitor processing can be performed in conjunction with the automatic tracking of the mobile phone holder 2 described above, or can be performed by a single mobile phone.

  As described above, the camera 10 of the mobile phone 1 acquires an image of a human face. The following two types of detection operations are performed based on the acquired image.

(1) Measurement of number of blinks The control unit 16 recognizes the human face and detects the position of the human eye from the coordinates of the contour. At this time, the eyes may be directly recognized by image processing. Thereafter, the control unit 16 measures the blink frequency from the acquired eye image. For example, the number of blinks per minute is measured. This is continuously performed, the moving average is calculated with a predetermined number of measurements (for example, 5), and the average number of blinks per unit time is measured.

  The controller 16 emits a warning sound from the speaker 12 using an internal sound source or the like when the number of blinks is outside a predetermined range (for example, ± 20%) from this average number. Note that information notifying the occurrence of abnormality may be transmitted so that a warning sound is emitted from the audio device of the automobile via the serial communication unit 20. When connected to the mobile phone holder 2, a warning sound may be emitted from an audio device of the automobile via the mobile phone holder 2. As the warning sound here, a sound notifying that an abnormality has occurred in the physical condition such as “Let's take a break” is emitted.

  Thus, the mobile phone 1 can measure human fatigue based on the number of blinks. This utilizes a phenomenon in which blinking is extremely reduced when fatigue is accumulated. Also, if there are too many blinks, it is highly likely that some kind of physical abnormality (such as dry eyes and pain) has occurred.

(2) Respiration frequency measurement The camera 10 of the mobile phone 1 acquires an image of a human face. The control unit 16 recognizes the human face and detects the positions of the human mouth and nose from the coordinates of the contour of the face. At this time, the mouth and nose may be directly recognized by image processing. Thereafter, the control unit 16 detects an infrared region of the acquired image. The camera 10 of the mobile phone 1 has sensitivity in the infrared region, and the control unit 16 can cut out the infrared region of the acquired image. The control unit 16 detects breathing from the extracted infrared region image. Since the temperature of human exhalation is higher than that of the surroundings, this exhalation can be detected by image recognition in an infrared region image. Thereafter, the control unit 16 measures the frequency of breathing. For example, the number of breaths per minute is measured. This is continuously performed, a moving average is calculated at a predetermined number of times of measurement (for example, 5 times), and the standard number of breaths per unit time is measured.

  The controller 16 emits a warning sound from the speaker 12 using an internal sound source or the like when the number of breaths falls outside the predetermined range from the standard number of breaths. Note that information notifying the occurrence of abnormality may be transmitted so that a warning sound is emitted from the audio device of the automobile via the serial communication unit 20. When connected to the mobile phone holder 2, a warning sound may be emitted from an audio device of the automobile via the mobile phone holder 2.

  As described above, the mobile phone 1 can measure the human tension based on the number of breaths. This utilizes the phenomenon that the number of breaths increases extremely as the degree of tension increases. Also, when the number of breathing is too small, a warning sound is generated because there is a high possibility that some kind of abnormality (such as dozing) has occurred.

  The driver monitor process can also be performed by the following two types of detection operations.

(1) Measurement of conversation reaction time The control unit 16 of the mobile phone 1 emits a conversation voice from the speaker 12 using an internal sound source or the like. After the conversation voice is uttered, the time until the reply is input to the microphone 11 is measured. This is continuously performed, a moving average is calculated at a predetermined number of measurements (for example, 5 times), and an average reaction time is measured.

  The control unit 16 emits a warning sound from the speaker 12 using an internal sound source or the like when the reaction time is outside the predetermined range from the average time. Note that information notifying the occurrence of abnormality may be transmitted so that a warning sound is emitted from the audio device of the automobile via the serial communication unit 20. When connected to the mobile phone holder 2, a warning sound may be emitted from an audio device of the automobile via the mobile phone holder 2.

  Thus, the mobile phone 1 can measure the human concentration level from the reaction time. This utilizes the phenomenon that the reaction time becomes extremely slow as the concentration decreases.

(2) Attitude variation measurement The acceleration sensor 15 of the mobile phone 1 inputs the detected acceleration to the control unit 16. The control unit 16 calculates the swinging direction of the mobile phone (change in posture of the mobile phone) from the input triaxial acceleration. Here, the attitude change of the automobile is estimated from the attitude change of the mobile phone. This is continuously performed, a moving average is calculated at a predetermined number of measurements (for example, 5 times), and an average posture variation is measured.

  The control unit 16 emits a warning sound from the speaker 12 using an internal sound source or the like when the posture variation is out of the predetermined range from this average. Note that information notifying the occurrence of abnormality may be transmitted so that a warning sound is emitted from the audio device of the automobile via the serial communication unit 20. When connected to the mobile phone holder 2, a warning sound may be emitted from an audio device of the automobile via the mobile phone holder 2.

  As described above, the mobile phone 1 can measure the human concentration level from the change in posture of the automobile. This uses the fact that the operation of the steering wheel, the accelerator, the brake, etc., varies when the degree of concentration decreases, and the car flutters.

  Any one of the above processes (blinking, breathing, conversation reaction, car posture) may be performed, or two or more processes may be performed simultaneously. Of course, all can be done at the same time. Even when two or more processes are performed at the same time, if any one of the physical conditions is abnormal (excessive fatigue, excessive tension, etc.), a warning may be issued. A warning may be issued when an abnormality occurs in an item. For example, when the number of breaths is too small and the posture change of the car becomes large, a warning is issued that the driver is sleeping and driving. Further, when the number of blinks is too small and the conversation reaction time is extremely slow, it is determined that the concentration is extremely low and the driver cannot concentrate on driving, and a warning is issued. Thereby, the body condition of the automobile driver can be measured with high accuracy.

  If an abnormality occurs, the center server or the like may be notified of the occurrence of the abnormality using a call function of the mobile phone. When the center server receives the notification of the occurrence of an abnormality, the center server may call the mobile phone, or may forward the notification of the occurrence of the abnormality to a contact point designated in advance.

  Next, operations of the mobile phone 1 and the mobile phone holder 2 of the present embodiment will be described in detail with reference to FIG. FIG. 4 is a flowchart showing the automatic tracking operation of the mobile phone 1 and the mobile phone holder 2.

  Connecting the cellular phone 1 to the cellular phone holder 2 is a trigger for starting this operation. In addition, it may be a trigger that the user instructs to read the program using the operation unit 14 of the mobile phone 1 after connection. First, the mobile phone 1 reads a program for performing driver monitoring processing (s10). That is, the camera 10 of the mobile phone 1 is activated to prepare for image shooting. The mobile phone holder 2 performs an initialization process (s20). In the initialization process, the drive positions of the drive unit 24A and the drive unit 24B are set to the center position of the drivable range. At this time, if the previous automatic tracking was performed and the positions of the drive unit 24A and the drive unit 24B were changed, the previous drive position is stored in a built-in storage unit (not shown) of the mobile phone holder 2. It may be set at this position. Alternatively, it may be stored in the storage unit of the mobile phone 1 and read out. When the initialization process is completed, the mobile phone holder 2 transmits information indicating that the standby is completed to the mobile phone 1 (s21), and the mobile phone 1 receives the information (s11).

  When the mobile phone 1 receives the information indicating that the standby has been completed, the mobile phone 1 acquires an image from the camera 10 and recognizes the human face by image processing (s12). At this time, if the face cannot be recognized, a warning is issued as occurrence of an abnormality (s13). As described above, the warning may be issued from the speaker 12 of the mobile phone 1 or from an audio device of the automobile.

  When the human face is recognized, the coordinates of the position within the camera shooting range are acquired (s14). Thereafter, the direction in which the pedestal is moved is calculated from the position coordinates, and a drive control signal is transmitted to the mobile phone holder 2 (s15). For example, when the human face is located on the left side of the camera shooting range, the circular platform 27 is rotated to the left side, and when the human face is located above the camera shooting range, the L-shaped table 28 is rotated upward. A drive control signal is transmitted. The movement amount may be moved by the minimum movement unit amount or may be moved by a predetermined movement amount. The mobile phone holder 2 receives this (s22), drives the drive unit 24A and the drive unit 24B by a predetermined amount, and moves the circular platform 27 and the L-shaped platform 28 (s23).

  Thereafter, the information that the movement has been completed is transmitted to the mobile phone 1 (s24). The mobile phone 1 receives this (s16), acquires an image again, and recognizes the human face (s17). Here, the position coordinate of the human face acquired last time is compared with the position coordinate of the human face acquired again, and the moving direction and movement deviation of the image are calculated (s18). The mobile phone 1 compares the movement deviation of the image with the movement amount of the previous stage, calculates the movement direction and movement amount of the stage so that the human face is at the center position of the image capturing range, and is driven again. A control signal is transmitted to the mobile phone holder 2 (s19). The mobile phone holder 2 receives this (s25), determines whether or not the movement amount is within the movable range of the table, and issues a warning as an abnormality if it is not within the movable range (s26). As described above, the warning may be issued from the audio device of the automobile, or may be issued from the speaker 12 of the mobile phone 1.

  If within the movable range, the drive unit 24A and the drive unit 24B are driven with the movement direction and movement amount indicated by the drive control signal, and the circular platform 27 and the L-shaped platform 28 are moved (s27). Thereafter, the processing returns to s24, and the processing is repeated so that the human face is accurately held at the center of the photographing range of the camera 10 so that the movement deviation is eliminated.

  As described above, the mobile phone 1 and the mobile phone holder 2 can automatically track the human face. Note that, as described above, image data is transmitted from the mobile phone 1 to the mobile phone holder 2, and the mobile phone holder 2 recognizes the tracking target image, and the tracking control is performed so that the shooting target comes near the center of the shooting range. You may make it do.

  Next, with reference to FIGS. 5 to 8, the operation of the program for performing the driver monitoring process will be described in detail. The driver monitoring process can be performed in conjunction with the automatic tracking of the mobile phone holder 2 described above, or can be performed by the mobile phone 1 alone. The driver monitoring process may be triggered by connecting the cellular phone 1 to the cellular phone holder 2 described above, or triggered by the user operating the operation unit 14 to instruct to read this program. It is good. Moreover, it is good also as a trigger that the user instruct | indicates to read this program, and also the acceleration sensor 15 detected the acceleration (the mobile telephone 1 was rock | fluctuated).

  FIG. 5 is a flowchart showing the fatigue measurement operation. First, an image is acquired from the camera 10 (s30), and a human face and eyes are recognized by image processing (s31). For eye recognition, the position is detected from the coordinates of the contour of the face, but the eyes may be directly recognized by image processing. As described above, recognition is performed based on a predetermined recognition algorithm.

  Thereafter, the blink frequency is measured from the acquired eye image (s32). For example, the number of blinks per minute is measured. For blinking, the area of the white eye is detected, and when the area falls below a predetermined ratio, it is determined that the eyes are closed, and the blink may be measured. Thereafter, moving average data is calculated with a predetermined number of measurements (for example, 5) (s33). If the number of measurements is less than 5 immediately after the start of the operation, the average data may be calculated based on the number of measurements, or the average data may not be calculated until the number is 5 or more.

  After calculating the moving average, it is determined whether 15 minutes have elapsed since the start of the operation (s34). If 15 minutes have not elapsed, the processing is repeated from the image acquisition (s34 → s30). When 15 minutes have elapsed, moving average data for 10 minutes after the start of operation is calculated as reference data (s35).

  The reference data is not limited to the moving average data of 5 to 15 minutes, but may be any time, but after the average number of blinks has settled (for example, after 5 minutes from the start of operation) It shall be calculated with data. This is because the physical condition of the driver is not settled immediately after the start of driving.

  Thereafter, it is determined whether or not the latest measurement data (latest moving average data) is within ± 20% of the reference data (s36). If it is within the range, the process is repeated from the image acquisition (s36 → s30). If it is outside ± 20% of the reference data, a warning is issued as being in a fatigue state (s37). Note that the criterion is not limited to ± 20%. What is necessary is just to adjust beforehand to the average reference | standard which can be judged that it is in a fatigue state according to the environment actually used.

  For the warning, an alarm sound is emitted from the speaker 12 using an internal sound source or the like. Note that information notifying the occurrence of abnormality may be transmitted so that a warning sound is emitted from the audio device of the automobile via the serial communication unit 20. When connected to the mobile phone holder 2, a warning sound may be emitted from an audio device of the automobile via the mobile phone holder 2.

  Next, FIG. 6 is a flowchart showing the tension measuring operation. First, an image is acquired from the camera 10 (s40), and the human face, mouth, and nose are recognized by image processing (s41). Regarding the recognition of the mouth and nose, the position is detected from the coordinates of the contour of the face, but the mouth and nose may be directly recognized by image processing.

  Thereafter, an infrared region is cut out from the acquired image (s42). As described above, since the camera 10 of the mobile phone 1 has sensitivity in the infrared region, the image of this region is acquired. The respiratory rate is measured from the acquired image (s43). For example, the number of breaths per minute is measured. The number of breaths is measured by detecting exhalation. Since the temperature of human exhalation is higher than that of the surroundings, this exhalation can be detected by image recognition in an infrared region image. The number of breaths is one per breath. Thereafter, moving average data is calculated with a predetermined number of measurements (for example, 5 times) (s44). If the number of measurements is less than 5 immediately after the start of the operation, the average data may be calculated based on the number of measurements, or the average data may not be calculated until 5 or more times.

  After calculating the moving average, it is determined whether 15 minutes have passed since the operation started (s45). If 15 minutes have not elapsed, the processing is repeated from the image acquisition (s45 → s40). When 15 minutes have elapsed, moving average data for 10 minutes after the start of operation is calculated as reference data (s46).

  Even in this operation, the reference data is not limited to moving average data of 5 to 15 minutes. Thereafter, it is determined whether or not the latest measurement data (latest moving average data) is within ± 20% of the reference data (s47). If it is within the range, the process is repeated from the image acquisition (s47 → s40). If it is outside ± 20% of the reference data, a warning is issued as being in tension (s48). Note that the criterion is not limited to ± 20%. What is necessary is just to adjust to the average reference | standard which can be judged that it is in the tension state according to the environment actually used. The warning may be emitted from the speaker 12 as described above, or a warning sound may be emitted from an audio device of the automobile.

  Next, FIG. 7 is a flowchart showing the conversation reaction degree measurement operation. First, a conversation voice is output using the speaker 12 (s50). The conversation voice may be any voice, but for example, a voice for asking “Is it all right?” Is issued. A reply voice from the driver is input for the voice conversation output for the inquiry (s51). If there is an input of a predetermined level or more from the microphone 11, it is determined as a response voice, but a specific voice (for example, “Yes”) may be determined as a response voice by voice recognition. If there is no response voice from the driver, a warning may be issued or this operation may be terminated.

  Thereafter, the time from when the conversation voice is output until the response voice is input is measured (s52), and the moving average data is calculated with a predetermined number of measurements (for example, five times) (s53). If the number of measurements is less than 5 immediately after the start of the operation, the average data may be calculated based on the number of measurements, or the average data may not be calculated until 5 or more times.

  After calculating the moving average, it is determined whether or not 15 minutes have elapsed since the start of the operation (s54). If 15 minutes have not elapsed, the process is repeated from the conversation voice output (s54 → s50). When 15 minutes have elapsed, moving average data for 10 minutes after the start of operation is calculated as reference data (s55). Even in this operation, the reference data is not limited to moving average data of 5 to 15 minutes.

  Thereafter, it is determined whether or not the latest measurement data (latest moving average data) is within ± 20% of the reference data (s56). If within the range, the process is repeated from the conversation voice output (s56 → s50). . If it is outside ± 20% of the reference data, a warning is issued (s57). Note that the criterion is not limited to ± 20%. It may be adjusted in consideration of the average conversational response variation according to the actual environment. The warning may be emitted from the speaker 12 as described above, or a warning sound may be emitted from an audio device of the automobile.

  Next, FIG. 8 is a flowchart showing the posture variation measuring operation. First, acceleration information is acquired from the acceleration sensor 15 (s60), and the mobile phone swing (posture fluctuation) is calculated (s61). Since the acceleration information can be detected in three axes (X, Y, Z), the orientation change direction of the mobile phone and its acceleration are calculated. The attitude change of the automobile is estimated and measured from the attitude change of the mobile phone (s62). When the mobile phone 1 is connected to the mobile phone holder 2, current mobile phone orientation information is acquired from the mobile phone holder 2 and associated with the traveling direction of the automobile. The mobile phone holder 2 is provided in a vehicle so that the mobile phone 1 faces in a direction opposite to the traveling direction of the vehicle, and associates the direction of the mobile phone 1 and the vehicle from the moving direction of the drive unit 24A and the drive unit 24B. .

  If the mobile phone 1 is not connected to the mobile phone holder 2, it is assumed that the mobile phone 1 is provided in a direction opposite to the traveling direction of the automobile, and the orientation of the mobile phone 1 and the automobile may be associated with each other. . Note that the user may manually input the direction of the mobile phone.

  In this way, the attitude variation of the automobile is measured, and moving average data is calculated with a predetermined number of measurements (for example, 5 times) (s63). If the number of measurements is less than 5 immediately after the start of the operation, the average data may be calculated based on the number of measurements, or the average data may not be calculated until 5 or more times.

  After calculating the moving average, it is determined whether 15 minutes have passed since the operation started (s64). If 15 minutes have not elapsed, the process is repeated from the acceleration information acquisition (s64 → s60). When 15 minutes have elapsed, moving average data for 10 minutes after the start of operation is calculated as reference data (s65). Even in this operation, the reference data is not limited to moving average data of 5 to 15 minutes.

  Thereafter, it is determined whether or not the latest measurement data (latest moving average data) is within ± 20% of the reference data (s66). If it is within the range, the process is repeated from acquisition of acceleration information (s66 → s60). . If it is outside ± 20% of the reference data, it is determined that the attitude of the vehicle is fluttering and a warning is issued (s67). Note that the criterion is not limited to ± 20%. It may be adjusted in advance so that it can be determined that the automobile is fluttering according to the environment in which it is actually used. The warning may be emitted from the speaker 12 as described above, or a warning sound may be emitted from an audio device of the automobile.

  Any one of the action items described above (fatigue level measurement action, tension level measurement action, conversational reactivity measurement action, and posture fluctuation measurement action) may be performed, or two or more items may be simultaneously performed. You may make it perform. Of course, all can be done at the same time. Which operation is performed simultaneously may be manually set by the user or may be defined in advance in the program.

  Even when two or more movements are performed simultaneously, if any one of the physical conditions is abnormal (excessive fatigue, excessive tension, reduced concentration due to delay in conversation reaction, etc.), a warning is issued. A warning may be issued when an abnormality occurs in a plurality of items. The warning is not limited to voice. A warning screen may be displayed on the display unit 13 to notify the driver of an abnormality in the physical condition.

  In the present embodiment, an example is shown in which the control unit 16 performs various processes by reading out a program. However, the control unit 16 may be realized by providing a dedicated IC chip set for performing these processes in the mobile phone. Accordingly, the driver monitoring process can be performed without imposing a burden on the control unit of the mobile phone.

  In the present embodiment, an example is shown in which the mobile phone 1 and the mobile phone holder 2 are connected, and the mobile phone holder 2 changes the orientation so that the human face is recognized at the center of the shooting range of the camera 10. However, when the camera 10 of the mobile phone 1 has a rotation mechanism alone, the control unit 16 may control so that the camera 10 faces the driver's face.

External view of a mobile phone according to an embodiment of the present invention External view of a mobile phone holder according to an embodiment of the present invention Block diagram of mobile phone and mobile phone holder Flow chart showing automatic tracking operation of mobile phone and holder for mobile phone Flow chart showing fatigue measurement operation Flow chart showing tension measurement operation Flow chart showing conversation reactivity measurement operation Flow chart showing posture fluctuation measurement operation

Explanation of symbols

1-mobile phone 2-mobile phone holder 10-camera 11-microphone 12-speaker 15-acceleration sensor 24-drive unit

Claims (13)

Image acquisition means for acquiring an image of a human face;
Image analysis means for detecting the human body condition from the image acquired by the image acquisition means;
An abnormality notification means for judging that the detected physical condition is abnormal when it falls outside a predetermined standard and issuing a warning, or outputting information indicating that a warning is issued to the outside;
Mobile phone equipped with.   The mobile phone according to claim 1, wherein the image analysis unit detects a physical state by detecting the number of blinks or the number of breaths per predetermined time. An audio output unit that emits conversational audio;
Voice input means for inputting human voice;
2. The conversation reaction detecting means for detecting a body state by measuring a time from when a voice output unit emits a conversation voice until a human voice is input to the voice input means. 2. The mobile phone according to 2. An acceleration sensor that detects acceleration in three axial directions;
Driving condition determining means for determining the driving condition of the automobile vehicle from the acceleration detected by the acceleration sensor,
4. The mobile phone according to claim 1, wherein the abnormality notifying unit determines that the abnormality is detected when the driving state detected by the driving state determining unit is out of a predetermined standard. The number of blinks per predetermined time detected by the image analysis means, the number of breaths, the time from when the conversational sound detected by the conversation reaction detection means is emitted until the human voice is input, and the driving condition determination means Detect at least two of the driving conditions
5. The mobile phone according to claim 2, wherein the abnormality notification unit issues a warning when it is determined that an abnormality has occurred in any one item. A base for fixing the mobile phone;
A data communication unit for performing data communication with the mobile phone;
A drive unit for rotating the pedestal;
A control unit for controlling a rotation angle of the driving unit;
A mobile phone holder comprising:
The mobile phone holder for controlling the drive unit based on a drive control signal input from the mobile phone via the data communication unit. A base for fixing the mobile phone;
A data communication unit for performing data communication with the mobile phone;
A drive unit for rotating the pedestal;
A control unit for controlling a rotation angle of the driving unit;
A mobile phone holder comprising:
The control unit is a mobile phone holder that controls a rotation angle so that a specific image received from the mobile phone via the data communication unit is displayed at the center of the mobile phone screen. On the mobile phone
Image acquisition procedure to acquire the image of the human face,
A physical state detection procedure for detecting the human physical state from the image acquired in the image acquisition procedure;
An abnormality notification procedure for issuing a warning when the detected physical condition falls outside a predetermined standard, or outputting information indicating that a warning is issued to the outside,
A mobile phone program that runs The physical condition detection procedure includes a procedure of detecting a physical condition by detecting the number of blinks per predetermined time or the number of breaths, or measuring the time from when a voice is spoken until a human voice is input. ,
The mobile phone program according to claim 8. An acceleration detection procedure for detecting acceleration in three axis directions;
Driving situation determination procedure for determining the driving situation of an automobile vehicle from the acceleration detected in the acceleration detection procedure
Is executed further,
10. The mobile phone program according to claim 8, wherein the abnormality notification procedure includes a procedure of determining an abnormality when the driving situation detected in the driving situation judging procedure is out of a predetermined standard. Built into a mobile phone,
Image acquisition procedure to acquire the image of the human face,
A physical state detection procedure for detecting the human physical state from the image acquired in the image acquisition procedure;
An abnormality notification procedure for issuing a warning when the detected physical condition falls outside a predetermined standard, or outputting information indicating that a warning is issued to the outside,
The semiconductor element which recorded the program for the cellular phone for executing.   The physical condition detection procedure includes a procedure of detecting a physical condition by detecting the number of blinks per predetermined time or the number of breaths, or measuring the time from when a voice is spoken until a human voice is input. The semiconductor device according to claim 11. An acceleration detection procedure for detecting acceleration in three axis directions;
Driving situation determination procedure for determining the driving situation of an automobile vehicle from the acceleration detected in the acceleration detection procedure
Record the program that will execute
The semiconductor element according to claim 11, wherein the abnormality notification procedure includes a procedure for determining that the abnormality is detected when the driving condition detected in the driving condition determination procedure is out of a predetermined reference.

Images