JP2005338934A - In-vehicle communication device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prompt a driver to communicate with passengers when the mental condition of the driver becomes abnormal, and to prevent the continuation of the abnormal state of the driver in the driving and traveling of a vehicle. <P>SOLUTION: This in-vehicle communication device is provided with an image pickup means 1 for picking up the image of the face of a driver who drives a vehicle 100 and a microcomputer 2 for detecting the mental condition of the driver, and for deciding whether or not the mental condition of the driver is abnormal on the basis of the detected mental condition of the driver, and for synthesizing a video signal to be inputted from external equipment with the video signal of the face of the driver imaged by the image pickup means 1 when it is decided that the mental condition of the driver is abnormal, and for making a monitor 3 display the video based on the composited video signal. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an in-vehicle communication device that detects a driver's mind-body condition and notifies the detected mind-body condition to an occupant other than the driver.

  In recent years, a nap driving that a driver falls into has become a big social problem, and an apparatus for preventing the napping driving has been devised and implemented. For example, a drowsy driving prevention system has been devised that enables an operator in a remote location to reliably grasp the driver's drowsiness state and provide sufficient safety guidance to the driver (see Patent Document 1). .)

  The doze driving prevention system described in Patent Document 1 includes a vehicle management system that is a master station and a vehicle system that is a slave station. The vehicle-side system includes data transmission / reception means for performing data communication with the vehicle management-side system, image input means for inputting facial expressions of the driver of the vehicle as image data, and whether or not a transmission condition for the image data is satisfied. The transmission condition determining means for determining, the warning means for giving a warning to alert the driver, and the warning operating means for operating the warning means based on the operation signal from the vehicle management system are configured. On the other hand, the vehicle management side system includes a data transmission / reception unit that performs data communication with a plurality of vehicle side systems, an image data display unit that displays image data transmitted from the vehicle side system, and a vehicle side system via the data transmission / reception unit. And remote operating means for remotely operating the warning means.

  According to the technology having such a configuration, when a transmission condition for image data is satisfied in the vehicle side system, the facial expression of the driver is input as image data, and this image data is automatically input to the vehicle management side system. Sent to. Then, in the vehicle management system, the image data transmitted from the vehicle system is displayed, and based on this display data, it is determined whether the driver is fatigued, that is, whether he / she is likely to fall asleep. To remotely activate the warning means of the vehicle side system.

  Here, the vehicle-side system refers to the map based on the current time, sets the image data transmission time, and when the set image data transmission time is reached, the image data transmission condition is satisfied. Judge. At this time, a transmission instruction signal is transmitted from the vehicle management side system to the vehicle side system, and it is determined that the transmission condition of the image data is satisfied in the vehicle side system. With this configuration, the vehicle management side system transmits an image data transmission instruction command to the vehicle side system when the driver's situation is to be confirmed, and the image data transmission condition is satisfied in the vehicle side system. It is judged.

Further, the vehicle-side system measures the current position of the vehicle and determines whether or not the vehicle is traveling on a specific route with reference to the map based on the measured current position of the vehicle. It is determined that the image data transmission condition is satisfied every time a predetermined time elapses after traveling on the specific route and starting traveling on the specific route. Then, the vehicle side system detects the traveling speed of the vehicle, determines whether or not the vehicle is monotonous by referring to the map based on the detected vehicle speed, and is in monotonous traveling and starts monotonous traveling. It is determined that the transmission condition of the image data is satisfied every predetermined time. In this way, the act of the driver communicating with others has an effect of correcting other psychosomatic abnormal states as well as falling asleep.
Japanese Patent Laid-Open No. 10-111994

  The doze driving prevention system disclosed in Patent Document 1 described above transmits a driver's face image to an operator at a remote place when the driver is likely to fall asleep during a time period or scene, and the transmitted face image is transmitted to the operator. This is a system that judges and warns the driver.

  However, when it is time or scene where a nap is likely to occur, the driver's face image is transmitted to an operator at a remote location to determine whether or not the driver is asleep and issue an alarm. Therefore, it is necessary to monitor multiple driver's face images at the same time, there is a possibility that it will actually fall into some abnormal situation, overlook the driver who is asleep, and lack of certainty is there. That is, even if there are other drivers who are really dozing, that is, in an abnormal state, there is a possibility that the operator does not know that and monitors the situation of other drivers.

  In addition, since the operator determines the dozing state only by the transmitted face image, there is a problem that the operator is largely dependent on the subjectivity of the operator, and there is a difference between the operators in the criteria for determining that the operator is dozing. . In addition, in order to unify the operator's judgment criteria, it is necessary to educate the operator professionally, so that there is a problem that a great amount of cost is required before the system is operated.

  Therefore, the present invention has been devised to solve the above-described problems, and in-vehicle communication between a passenger and a driver who are in the same vehicle is performed according to the mental and physical state of the driver. It is an object of the present invention to provide an in-vehicle communication device that can smoothly support a driver without obstructing the driving operation.

  The present invention is an in-vehicle communication device mounted on a vehicle, the imaging means for imaging the face of the driver who performs the driving operation of the vehicle, the mind-body state detecting means for detecting the mind-body state of the driver, and the mind-body state Based on the driver's mind-body state detected by the detector, the determination means for determining whether the driver's mind-body state is abnormal, the input means for inputting a video signal supplied from an external device, and the input means Display means for displaying video based on the input video signal.

  In such a configuration, when the determination unit determines that the driver's mind and body condition is abnormal by the determination unit, the control unit detects the video signal input by the input unit and the driver's face imaged by the imaging unit. The above-mentioned problem is solved by causing the occupant other than the driver to visually recognize the video of the driver's face by combining the video signal and controlling the video signal to be displayed on the display means.

  According to the in-vehicle communication device according to the present invention, when the driver's mind and body state is abnormal, the driver's face is visually recognized by an occupant other than the driver, so the occupant is viewing other images. Even in this case, it is possible to smoothly support in-vehicle communication between an occupant and a driver who are in the same vehicle without obstructing the driving operation of the driver.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
As shown in FIGS. 1 and 2, the in-vehicle communication device 10 according to the first embodiment performs a driving operation of the vehicle 100 from the camera 1 that captures the face of the driver and the face image data captured by the camera 1. In addition to the microcomputer 2 that detects the mind and body state of the driver to be performed and determines whether or not the vehicle 100 is suitable for driving, the map information of the navigation system, the video source such as TV, video, and the like are output and displayed. And a monitor 3 for outputting and displaying the determination result from the computer 2.

  The camera 1 is an image pickup means, and is a small-sized image pickup device including, for example, a CCD (Charge Coupled Device) image pickup device. As shown in FIG. 2, the camera 1 is installed on the instrument panel 101 of the vehicle 100 at a position where a substantially front face of the driver's face can be imaged. The driver's face image data captured by the camera 1 is sent to the microcomputer 2 as digital image data.

  The microcomputer 2 is a control device that comprehensively controls the operation of the in-vehicle communication device 10. The microcomputer 2 detects the mind-body state of the driver as described above, and the driving according to the detected state. It functions as a determination unit that determines whether or not it is possible, and executes based on predetermined software. The microcomputer 2 also displays video from an external device such as map information from a navigation system (not shown), a television signal received by a television tuner, a video signal sent from a video playback device, and a video game signal from a video game machine. An external video input port (not shown) for inputting a signal and a video output port (not shown) for performing predetermined image processing using the video signal input from the external video input port and outputting the image to the monitor 3 are provided.

  The microcomputer 2 normally functions as input means, and is supplied with the video signal from the external device as described above via the external video input port. The microcomputer 2 performs predetermined image processing on the video signal so that it can be output and displayed on the monitor 3, and outputs the video signal to the monitor 3 via the video output port.

  When the microcomputer 2 determines from the face image data sent from the camera 1 that the driver's mind and body condition is not suitable for driving the vehicle 100, the face image data can be displayed on the monitor 3. Convert to a clear video signal. Then, it is synthesized with the video signal output in the normal state and sent to the monitor 3 via the video output port, or it is not synthesized as it is, but instead of the video signal from the outside, it is monitored via the video output port. The video switching control is performed so that the image is transmitted to the terminal 3. Thereby, the microcomputer 2 functions as a control means.

  The monitor 3 includes, for example, a liquid crystal display (LCD) that can display 640 pixels in the horizontal direction and 480 pixels in the vertical direction. The monitor 3 is connected to the video output port of the microcomputer 2 described above, and outputs and displays a video signal input via the video output port. It is not limited to the type of the monitor 3 and may be thin enough to be mounted on the vehicle 100. For example, a self-luminous type that does not require a backlight device such as an organic EL display or an inorganic EL display. A thin display or the like may be used.

  The monitor 3 is installed at a position where the video information does not directly enter the driver's field of view and where a passenger other than the driver can see the video information output and displayed. For example, as shown in FIG. 2, the monitor 3 is installed on the back side of the front seat 102 of the vehicle 100 at a position where a passenger on the rear seat 103 can easily see.

  Usually, a passenger other than the driver can visually recognize information from a navigation system mounted on the vehicle 100, view a television or video, or play a video game via the monitor 3. . In addition, as described above, when the microcomputer 2 determines that the driver's mind and body state is not suitable for driving the vehicle 100, the monitor 3 sends the driver's face sent out from the microcomputer 2. Will be displayed to notify passengers other than the driver.

[Operation of In-Vehicle Communication Device 10]
Next, the processing operation of the vehicle communication device 10 configured as described above will be described using the flowchart shown in FIG.

  This process is started in response to turning on the microcomputer 2 by turning on the IGN switch of the vehicle 100, for example. First, in step S1, the driver's face is imaged by the camera 1. The face image data captured and acquired by the camera 1 is sent to the microcomputer 2.

  In the next step S2, the microcomputer 2 uses the face image data acquired in step S1 to start a driver abnormal state detection process for detecting the driver's mind and body state and detecting an abnormal state. This abnormal state detection process will be described in detail later.

  In the next step S <b> 3, the microcomputer 2 determines whether or not the driver's current mind and body state is suitable for driving the vehicle 100, according to the driver's mind and body state detected in step S <b> 2. If the microcomputer 2 determines that the mind and body state of the driver is a normal state suitable for driving the vehicle 100, the microcomputer 2 returns the process to step S <b> 1 and determines that the driver is in an abnormal state unsuitable for driving the vehicle 100. If so, the process proceeds to step S4. If the abnormal state of the driver is not detected and the process returns from step S3 to step S1, the microcomputer 2 assumes that the normal state is present, and sends the video signal from the external device to the monitor 3 from the video output port. Output. When no external device is connected to the external video input port, a black screen or a standby screen is displayed on the monitor 3.

  In the next step S4, the microcomputer 2 detects whether the video signal input from the external device via the video signal input port is supplied to the monitor 3 via the video output port. That is, the display state of the monitor 3 is detected.

  In the next step S5, the microcomputer 2 supplies the video signal input from the external device via the video signal input port to the monitor 3 via the video output port according to the detection result of step S4. If so, the process proceeds to step S6. If the video signal from the external device is not supplied to the monitor 3, the process proceeds to step S7.

  In step S6, since the video signal is input from the external device via the video signal input port, the microcomputer 2 interrupts the face image data captured by the camera 1 and combines the video signal. Determine the size. The determination processing of the interrupt image size of the face image data will be described in detail later.

  On the other hand, in step S <b> 7, the microcomputer 2 converts the face image data captured by the camera 1 into a video signal to be supplied to the monitor 3. At this time, if a video signal is supplied from an external device via the video signal input port, the video signal generated from the face image data so as to have the image size determined in step S6 is added to the video signal. Combined and sent to the monitor 3. On the other hand, if it is determined in step S5 that no video signal is input from the external device, the face image data is sent to the monitor 3 as it is without performing a composition process.

"Driver abnormal state detection process"
The driver abnormal state detection process executed by the microcomputer 2 in step S2 will be described. The abnormal state of the driver is a state in which a calm judgment different from usual cannot be made such as, for example, falling asleep, looking aside, sloppy, anger (irritated), and excessive tension. As a method for detecting these abnormal states, for example, as shown below, information obtained by performing image processing on the driver's face image data captured by the camera 1 and input to the microcomputer 2 is used. A method of making a determination based on this, a method of detecting a driver's biological signal, and making a determination based on this biological signal can be used.

  Examples of methods for detecting an abnormal state of a driver by image processing from face image data include a method for detecting a drowsiness state, a method for detecting a side look, a method for detecting abusive driving, and a stress state such as anger rather than facial expression recognition. And the like. Among such known methods, the microcomputer 2 may use any method.

  As an example, a method for detecting the driver's dozing state will be described. First, the microcomputer 2 extracts the feature amount from the face image data by image processing to detect the position of the driver's eyes, and further measures the eye closing time, which is the time during which the eyes are closed.

  Normally, eye closure time can be broadly classified into short-term, middle-time, and long-term eye closures, and will appear with different tendencies during arousal, which is a normal state of mind and body, and during hypotension, which is a state of mind and body abnormality. . Although there are some individual differences, normally, in the normal state, the eye is closed for a short time and the eye is closed frequently for a short time. In addition, middle-time eye closure occurs most frequently when the person is about to fall asleep. Long-term closed eyes do not occur unless they are in good condition. By taking into account the general tendency of eye closure time, taking into account the length of each eye closure time, the driver's condition, the driver's environment, etc. The detection of the dozing state can be executed with high accuracy.

  Various methods have been devised and implemented for detecting the driver's abnormal state from the driver's biological signal, and any known method may be used.

  For example, the driver's biological signals include the driver's heart rate, brain waves, sweat volume, head vibration, skin potential, skin electrical resistance, blink, body acceleration, and body pressure such as the pressure between the driver and the back seat. The driver's tension can be detected by the measured values. The abnormal state of the driver's mind and body can be detected based on the degree of tension obtained from the biological signal.

  Specifically, as shown in FIG. 4, the microcomputer 2 detects a biological signal such as a driver's heart rate from the detection electrodes 104a and 104b provided on the steering handle 104, and sets the detected biological signal level. Accordingly, an abnormal state is determined. Thereby, the microcomputer 2 functions as a biological information detection means.

  In fact, the microcomputer 2 does not detect all abnormal states such as heart rate, brain wave, sweating amount, snooze, aside, messy, anger (irritated), excessive tension, but detects any abnormal state If it is possible, the process proceeds to step S3, and it is determined whether or not the state is suitable for driving. Therefore, the microcomputer 2 only needs to be able to execute at least one of a method for detecting an abnormal state from face image data and a method for detecting an abnormal state from a biological signal. When these two methods are performed simultaneously, a processing load is applied, but the accuracy can be improved. In addition to the method described above, a method for detecting an abnormal state of the driver can be flexibly taken in and dealt with.

"Face image data interrupt size determination process"
Next, the process for determining the interrupt size of face image data executed in step S6 will be described.

  For example, the microcomputer 2 determines whether the driver's abnormal state is one of three levels of large level, medium level, and small level in the driver's abnormal state detection and determination process in steps S2 and S3. To do. When the abnormal state is determined for each level as described above, the urgency and quickness are improved as the abnormal state detected and determined in steps S2 and S3 changes from small level to medium level to large level with time change. Becomes higher.

  For example, it is assumed that a map image MG generated from the map information input to the microcomputer 2 from the navigation system as shown in FIG. In steps S2 and S3, when the microcomputer 2 determines that the detected abnormal state of the driver is at a low level, as shown in FIG. 5B, the microcomputer 2 sets the size of the face image FG to the image of the map image MG. The size is determined to be, for example, about 1/4 or less, and is interrupted on the map image MG. Furthermore, when the microcomputer 2 determines that the abnormal state of the driver is the middle level, the microcomputer 2 determines the size of the face image FG to be equal to the size of the map image MG, as shown in FIG. , Interrupt the map image MG. Further, when the microcomputer 2 determines that the driver's abnormal state is a large level, the microcomputer 2 displays the face image FG on the substantially entire surface of the monitor 3 as shown in FIG. For example, it is determined to be about 1/4 or less of the face image FG. That is, it is an image in a state where the map image MG and the face image FG in FIG.

  If the abnormal state continues, the size of the face image FG is changed as shown in FIG. 5 (b) → FIG. 5 (c) → FIG. 5 (d) even if the level of the abnormal state does not change. The abnormal state may be emphasized. Furthermore, when the display state of the monitor 3 continues as shown in FIG. 5D, the map image MG is not displayed at all as shown in FIG. Only the face image FG may be displayed on the entire surface to emphasize the abnormal state.

  Note that, as an example, the case where the map image MG generated from the map information is displayed on the monitor 3 has been described. However, the microcomputer 2 has been described above regardless of the type of the video signal input from the external device. The interrupt size of the face image data is determined in the same manner as the method, and output from the monitor 3 is displayed.

[Effect of the first embodiment]
As described above in detail, according to the in-vehicle communication device 10 according to the first embodiment to which the present invention is applied, an image viewed by an occupant other than the driver in a normal state is displayed on the mind and body state of the driver. Accordingly, the video whose image size is determined is forcibly interrupted, so that the occupant can be notified of the state of mind and body of the driver immediately. Accordingly, communication between the occupant and the driver can be smoothly induced to prevent the driver from continuing the abnormal state. In other words, even if the passenger enjoys another video that is not a face image, the face image is interrupted in all or part of the video, thereby inducing smooth communication between the passenger and the driver. be able to.

  In addition, according to the in-vehicle communication device 10, the face image data is image-processed to detect at least one of the driver's dozing, looking aside, abusive state, anger (irritated), and excessive tension, When it is determined that the vehicle is in a state, it is determined that the driver is in an abnormal state, so that the abnormal state of the driver can be accurately detected.

  Furthermore, according to this in-vehicle communication device 10, biological signals such as a driver's heartbeat, brain waves, and sweating amount are collected to determine a state in which the driver is abnormal. Can be detected.

  Furthermore, according to this in-vehicle communication device 10, when the level of the driver's abnormal state is high, the driver's face image size is made larger than the original video information, and the level of the driver's abnormal state is increased. Is smaller than the original video information, the driver's face image size is made smaller, so communication according to the level of the abnormal state can be smoothly induced.

[Second Embodiment]
Next, an in-vehicle communication device 20 according to a second embodiment to which the present invention is applied will be described.

  As shown in FIG. 6, the in-vehicle communication apparatus 20 includes a camera 1, a microcomputer 12, and a monitor 3, and the microcomputer 2 of the in-vehicle communication apparatus 10 described with reference to FIGS. The configuration is exactly the same except that is replaced with the microcomputer 12. Therefore, the same functional parts as those in the in-vehicle communication device 10 are denoted by the same reference numerals and description thereof is omitted.

  The microcomputer 12 detects the driver's mind-body state included in the microcomputer 2 described above, and in addition to all functions including a driving state detection function for detecting whether or not driving is possible, driving for specifying the driver. And a character information generating function for generating character information to be output and displayed on the monitor 3 from the driver identifying function, the identified driver personal information, and the driver's mind and body state. The microcomputer 2 is provided with each part of the driver person specific | specification part 21, the driving | running state detection part 22, and the character information generation part 23 by performing these various functions based on software.

  In the processing operation of the vehicle communication device 20, as shown in FIG. 7, first, in step S <b> 11, the driver's face is imaged by the camera 1, and face image data is sent to the microcomputer 12.

  In the next step S12, the driver person specifying unit 21 of the microcomputer 12 specifies the driver who is driving the vehicle 100 among the pre-registered persons, and acquires the person information of the specified driver. The driver identification process to be executed is executed. The driver specifying process will be described later.

  In the next step S13, the driving state detection unit 22 of the microcomputer 12 starts a driver abnormal state detection process for detecting the driver's mind and body state. Since this abnormal state detection process has been described in the first embodiment, a description thereof will be omitted.

  In the next step S <b> 14, the microcomputer 12 determines whether or not the driver's current mind-body state is a state suitable for driving the vehicle 100 according to the detected mind-body state of the driver. If it is determined that the driver's mind and body state is suitable for driving the vehicle 100, the process returns to step S12. If the driver is determined to be in an abnormal state unsuitable for driving, the process proceeds to step S15. Proceed to.

  In step S15, the microcomputer 12 detects whether the video signal input from the external device via the video signal input port is supplied to the monitor 3 via the video output port. Detect the display state.

  In the next step S16, the microcomputer 12 supplies the video signal input from the external device via the video signal input port to the monitor 3 via the video output port according to the detection result of step S15. If so, the process proceeds to step S17. If the video signal from the external device is not supplied to the monitor 3, the process proceeds to step S18.

  In step S <b> 17, the microcomputer 12 interrupts the face image data captured by the camera 1 in the image created by the video signal in response to the video signal being input from the external device via the video signal input port. Determine the size when combining. Since the process for determining the interrupt size of the face image data has been described in the first embodiment, a description thereof will be omitted.

  In the next step S18, the character information generating unit 23 of the microcomputer 12 uses the personal information of the driver acquired in step S12 and the driver's mind and body state detected and determined in steps S13 and S14 on the monitor 3. Character information, which is a character string incorporating a character indicating the driver for output and display on, is generated. This character information generation process will be described later.

  In the next step S <b> 19, the microcomputer 12 converts the face image data captured by the camera 1 into a video signal to be supplied to the monitor 3. At this time, if a video signal is supplied from an external device via the video signal input port, the video signal generated from the face image data is combined with the video signal so as to have the image size determined in step S17. Further, the character information generated in step S18 is added and sent to the monitor 3.

  When no video signal is input from the external device, the character information generated in step S18 is added to the video signal generated from the face image data without performing the video signal synthesis process, and the monitor 3 is added. To send.

"Driver specific processing"
Next, driver identification processing executed by the driver person identification unit 21 in step S12 will be described.

  There are various methods for specifying the driver by the driver person specifying unit 21. In this example, the face image data captured by the camera 1 and the features of the face image registered in advance in the database are used. The driver is identified by checking the information. When the driver is specified, the microcomputer 12 reads and acquires the driver's personal information specified from the database. In addition, by using the biometric information unique to the driver, such as fingerprint information and voiceprint information, without using the face image data in this way, the microcomputer 12 performs verification processing to obtain driving information. The person may be specified.

  And if the driver | operator is specified, the microcomputer 12 will read and acquire the driver | operator's personal information specified from the database. In this case, since it is not necessary to acquire face image data as in the flowchart shown in FIG. 7, the order of step S11 and step S12 may be switched.

  In any of the above-described methods, the driver is identified almost automatically by the microcomputer 12 after acquiring biological information. For example, a person who can be a driver is registered in a switch or the like. A manual authentication method in which the person who becomes the driver inputs the switch when driving to notify the microcomputer 12 of the driver's personal information can also be applied.

"Character information generation process"
Next, the character information generation process executed in step S18 will be described.

  In the character information generation process of step S18, the microcomputer 12 uses a person name or a name that can identify a person (nickname, family name such as father or mother) from the person information obtained by the driver identification process of step S12. In accordance with the level of the driver's abnormal state determined by the driver's abnormal state detection and determination process in steps S13 and S14, character information incorporating the determined person name and name is generated.

  For example, when the abnormal state is at a low level, the microcomputer 12 provides character information such as “Let's talk to Mr. XX” that prompts the passenger who is watching the monitor 3 to talk to the driver. The character information is generated, added to the video information, and output and displayed on the monitor 3.

  Also, when the abnormal state is medium level, the microcomputer 12 is bored to call attention to the passengers who are watching the monitor 3. “Please be careful. Is generated, added to the video information, and output and displayed on the monitor 3.

  Furthermore, when the abnormal state is at a high level, the microcomputer 12 is likely to sleep “Mr. XX” who is directly paying attention to the passenger watching the monitor 3. Please generate character information such as "Please add the character information to the video information and display it on the monitor 3".

  Specifically, the character information generation unit 23 adds the character information MJ to the video information as shown in FIGS. 8A to 8F and outputs and displays it on the monitor 3. 8A, 8B, and 8C are examples in the case where a video signal from an external device, here, a video signal based on the map information MG from the navigation system is input to the microcomputer 12. FIG. FIG. 8D, FIG. 8E, and FIG. 8F show examples in which the microcomputer 12 does not receive a video signal input from an external device.

  In addition, the character information generation unit 23 changes the level of the driver's abnormal state from small level → medium level → high level to FIG. 8A → FIG. 8B → FIG. 8C. ) Or FIG. 8D → FIG. 8E → FIG. 8F, the image and character information output and displayed on the monitor 3 may be changed.

  If the microcomputer 12 cannot identify the driver in the driver specifying process in step S12, the microcomputer 12 changes the proper noun “XX” to a common noun such as “driver” or “driver” or adds a name. Instead, only character information corresponding to the level in the abnormal state may be generated.

[Effects of Second Embodiment]
As described above in detail, according to the in-vehicle communication device 20 according to the second embodiment, an image that is being viewed by a passenger other than the driver in a normal state is displayed according to the state of mind of the driver. Since the video whose size has been determined is interrupted and character information is added to the video, it is possible to accurately notify the passenger of the state of mind and body of the driver. In response to this, the occupant can prevent the driver's abnormal state from continuing by starting communication with the driver.

  Further, according to this in-vehicle communication device 20, since the driver's person is specified and the name of the specified person is included in the character information and presented to the passenger, further communication between the passenger and the driver is performed. Can be induced more smoothly.

  Furthermore, according to this in-vehicle communication device 20, since the person identification is performed by comparing the face image data acquired by the camera 1 with the image data stored in the database in advance, the person identification of the driver can be accurately performed. It can be carried out.

[Third Embodiment]
Next, the in-vehicle communication device 30 according to the third embodiment will be described with reference to FIGS. 9 and 10.

  As shown in FIGS. 9 and 10, the in-vehicle communication device 30 includes a camera 1, a microcomputer 2, a monitor 3, a microphone 4, and a speaker 5, and will be described with reference to FIGS. 1 and 2. The configuration is exactly the same except that the microphone 4 and the speaker 5 are added to the in-vehicle communication device 10. Therefore, the same functional parts as those in the in-vehicle communication device 10 are denoted by the same reference numerals and description thereof is omitted.

  The microphone 4 collects the voices of passengers other than the driver. For example, as shown in FIG. 10, the microphone 4 is installed on the back side of the front seat 102 at a position where it is easy to collect a voice uttered by a passenger on the rear seat 103.

  The speaker 5 outputs the passenger's voice collected by the microphone 4 to the driver via a voice transmission circuit (not shown). The speaker 5 is installed, for example, in the headrest 105 portion of the driver's seat where the driver is seated as shown in FIG.

  Next, the processing operation of the vehicle communication device 30 will be described using the flowchart shown in FIG.

  In step S <b> 21, the driver's face is imaged by the camera 1, and the acquired face image data is sent to the microcomputer 2.

  In the next step S22, the microcomputer 2 starts a driver abnormal state detection process for detecting the driver's mind and body state. Since this abnormal state detection process has been described in the first embodiment, a description thereof will be omitted.

  In the next step S <b> 23, the microcomputer 2 determines whether or not the current state of mind of the driver is suitable for driving the vehicle 100 according to the detected state of mind of the driver. If the microcomputer 2 determines that the driver's mind and body state is suitable for driving the vehicle 100, the microcomputer 2 proceeds to step S24, and if the microcomputer 2 determines that the driver is in an abnormal state not suitable for driving. Then, the process proceeds to step S25.

  In step S24, when it is determined that the state of mind of the driver is suitable for driving the vehicle 100, the sound transmission circuit connecting the microphone 4 and the speaker 5 is turned off to stop the function. Then, the process returns to step S21, and the video signal from the external device is output from the video output port to the monitor 3 assuming that the normal state.

  On the other hand, in step S25, when it is determined that the driver's mind and body state is not suitable for driving the vehicle 100, the voice transmission circuit that connects the microphone 4 and the speaker 5 is turned on, Is made active. Thereby, the passengers other than the driver are in a state in which their voices can be transmitted from the microphone 4 to the driver via the speaker 5. For example, the passenger can input a message for calling attention from the microphone 4 that the driver's mind and body condition is abnormal.

  In the next step S26, the microcomputer 2 detects whether the video signal input from the external device via the video signal input port is supplied to the monitor 3 via the video output port. Detect the display screen state.

  In the next step S27, the microcomputer 2 supplies the video signal input from the external device via the video signal input port to the monitor 3 via the video output port according to the detection result of step S26. If so, the process proceeds to step S28. If the video signal from the external device is not supplied to the monitor 3, the process proceeds to step S29.

  In step S28, the microcomputer 2 causes the face image data captured by the camera 1 to interrupt the image created by the video signal in response to the video signal being input from the external device via the video signal input port. Determine the size when compositing. Since the process for determining the interrupt size of the face image data has been described in the first embodiment, a description thereof will be omitted.

  On the other hand, in step S29, the microcomputer 2 converts the face image data captured by the camera 1 into a video signal to be supplied to the monitor 3. At this time, if a video signal is supplied from an external device via the video signal input port, the video signal generated from the face image data so as to have the image size determined in step S28 is added to the video signal. Are sent out to the monitor 3.

[Effect of the third embodiment]
As described above, according to the in-vehicle communication device 30 according to the third embodiment, a video whose image size is determined according to the mind and body state of the driver is forcibly displayed on a video that is being viewed by a passenger other than the driver. Since the microphone 4 and the speaker 5 that connect the passenger and the driver are electrically connected at the same time, the communication between the passenger and the driver can be further promoted.

[Fourth Embodiment]
Next, the in-vehicle communication device 40 according to the fourth embodiment will be described with reference to FIG.

  The in-vehicle communication device 40 includes a camera 1, a microcomputer 12, a monitor 3, a microphone 4, and a speaker 5. The microcomputer 2 of the in-vehicle communication device 30 described with reference to FIGS. The configuration is exactly the same except that is replaced with the microcomputer 12 of the in-vehicle communication device 20 described with reference to FIG. Therefore, the same function parts as those in the in-vehicle communication devices 20 and 30 are denoted by the same reference numerals and description thereof is omitted.

  The processing operation of the vehicle communication device 40 will be described with reference to the flowchart shown in FIG.

  First, in step S31, the driver's face is imaged by the camera 1, and the acquired face image data is sent to the microcomputer 12.

  In the next step S32, a driver specifying process for specifying a driver who is driving the vehicle 100 and acquiring personal information of the driver is executed. Since this driver specifying process has been described in the second embodiment, a description thereof will be omitted.

  In the next step S33, the microcomputer 12 starts a driver abnormal state detection process for detecting the driver's mind and body state. Since this abnormal state detection process has been described in the first embodiment, a description thereof will be omitted.

  In the next step S <b> 34, the microcomputer 12 determines whether or not the driver's current mind-body state is a state suitable for driving the vehicle 100, according to the detected mind-body state of the driver. If the microcomputer 12 determines that the state of mind of the driver is suitable for driving the vehicle 100, the microcomputer 12 proceeds to step S35, and if the microcomputer 12 determines that the abnormal state is not suitable for driving. Then, the process proceeds to step S36.

  In step S35, when it is determined that the state of mind of the driver is suitable for driving the vehicle 100, the voice transmission circuit connecting the microphone 4 and the speaker 5 is turned off to stop the function. Then, the process returns to step S32.

  On the other hand, in step S36, when it is determined that the driver's mind and body state is not suitable for driving the vehicle 100, the voice transmission circuit connecting the microphone 4 and the speaker 5 is turned on, and the function is performed. By setting the active state, passengers other than the driver can transmit their voices to the driver via the microphone 4 and the speaker 5.

  In the next step S37, the microcomputer 12 detects whether the video signal input from the external device via the video signal input port is supplied to the monitor 3 via the video output port. Detect the display screen state.

  In the next step S38, the microcomputer 12 supplies the video signal input from the external device via the video signal input port to the monitor 3 via the video output port according to the detection result of step S37. If so, the process proceeds to step S39. If the video signal from the external device is not supplied to the monitor 3, the process proceeds to step S40.

  In the next step S39, the microcomputer 12 interrupts and synthesizes the face image data captured by the camera 1 in response to the video signal being input from the external device via the video signal input port. Determine the size. The process for determining the interrupt size of the face image data has been described in the first embodiment, and is therefore omitted.

  In step S40, the microcomputer 12 outputs and displays on the monitor 3 the character information of the driver obtained in step S32 and the driver's mind and body state detected and determined in steps S33 and S34. Is generated. Since this character information generation process has been described in the second embodiment, a description thereof will be omitted.

  In step S <b> 41, the microcomputer 12 converts the face image data captured by the camera 1 into a video signal to be supplied to the monitor 3. At this time, if the video signal is supplied from the external device via the video signal input port, the video signal generated from the face image data is synthesized so as to have the image size determined in step S39, and The character information generated in step S40 is added and sent to the monitor 3.

  If no video signal is input from the external device, the character information generated in step S40 is added to the video signal generated from the face image data without performing the video signal synthesis process, and the monitor 3 is added. To send.

[Effect of Fourth Embodiment]
According to the in-vehicle communication device 40 according to the fourth embodiment as described above, an image whose image size is determined in accordance with the state of mind and body of the driver is inserted into an image that is being viewed by a passenger other than the driver. At the same time, by presenting character information and further turning on the voice transmission circuit, communication between the passenger and the driver can be promoted more reliably and accurately.

  The above-described embodiment is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and various modifications can be made depending on the design and the like as long as the technical idea according to the present invention is not deviated from this embodiment. Of course, it is possible to change.

It is the block diagram which showed the structure of the vehicle communication apparatus which concerns on 1st Embodiment to which this invention is applied. It is the figure which showed a mode that the vehicle communication apparatus which concerns on 1st Embodiment to which this invention was applied was installed in the vehicle. It is a flowchart for demonstrating the processing operation of the vehicle communication apparatus which concerns on 1st Embodiment to which this invention is applied. It is the figure which showed the steering handle with which the vehicle communication apparatus which concerns on 1st Embodiment to which this invention is applied is provided. FIGS. 5A to 5E are diagrams showing images displayed on the monitor in accordance with the difference in the abnormal level of the psychosomatic state. It is the figure which showed the structure of the vehicle communication apparatus which concerns on 2nd Embodiment to which this invention is applied. It is a flowchart for demonstrating the processing operation of the vehicle communication apparatus which concerns on 2nd Embodiment to which this invention is applied. FIGS. 8A to 8F are diagrams showing images to which character information displayed on the monitor is added according to the difference in the abnormal level of the mind and body state. It is the figure which showed the structure of the vehicle communication apparatus which concerns on 3rd Embodiment to which this invention is applied. It is the figure which showed a mode that the vehicle communication apparatus which concerns on 3rd Embodiment to which this invention was applied was installed in the vehicle. It is a flowchart for demonstrating the processing operation of the vehicle communication apparatus which concerns on 3rd Embodiment to which this invention is applied. It is the figure which showed the structure of the vehicle communication apparatus which concerns on 4th Embodiment to which this invention is applied. It is a flowchart for demonstrating the processing operation of the vehicle communication apparatus which concerns on 4th Embodiment to which this invention is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Camera 2 Microcomputer 3 Monitor 4 Microphone 5 Speaker 10, 20, 30, 40 In-vehicle communication apparatus 12 Microcomputer 21 Driver person specific part 22 Driving state detection part 23 Character information generation part 100 Vehicle 101 Instrument panel 102 Front part Seat 103 Rear seat 104 Steering handle 105 Headrest

Claims (8)

An in-vehicle communication device mounted on a vehicle,
Imaging means for imaging the face of the driver who performs the driving operation of the vehicle;
A psychosomatic state detection means for detecting the driver's psychosomatic state;
Determination means for determining whether or not the driver's mind-body condition is abnormal based on the mind-body condition of the driver detected by the mind-body condition detector;
Input means for inputting a video signal supplied from an external device;
Display means for displaying video based on the video signal input by the input means;
When it is determined by the determination means that the driver's mind and body condition is abnormal, the video signal input by the input means and the video signal of the driver's face imaged by the imaging means are combined, Control means for controlling the synthesized video signal to be displayed on the display means. An in-vehicle communication device comprising: The mind-body state detection means performs predetermined image processing on the driver's face image data imaged by the imaging means, and the driver's mind-body state falls asleep, looks aside, abusive, angry, excessive tension Detect either
The determination means determines that the driver's mind-body condition is abnormal when any mind-body condition is detected among the mind-body condition detection means: doze, look aside, sloppy condition, anger, and excessive tension. The in-vehicle communication device according to claim 1. The psychosomatic state detecting means further comprises biometric information detecting means for detecting biometric information of the driver and detecting any one of the driver's psychosomatic state such as dozing, looking aside, sloppy state, anger, and excessive tension. The determination means determines that the driver's mind-body condition is abnormal when any mind-body condition is detected among the mind-body condition detection means: doze, look aside, sloppy condition, anger, and excessive tension. The in-vehicle communication device according to claim 1. According to a determination result based on the driver's mind and body state by the determination means, further comprising: character information generating means for generating character information for notifying the driver's mind and body state;
The control means adds the character information generated by the character information generation means to the synthesized video signal, and controls the display means to display a video based on the video signal. Item 2. The in-vehicle communication device according to Item 1. A driver identification unit for identifying the driver;
The in-vehicle communication device according to claim 4, wherein the character information generating unit generates the character information of a character string incorporating a character indicating the driver specified by the driver specifying unit.   The driver identification means identifies the driver by comparing and collating face image data captured by the imaging means with face image data on a database stored in advance. 5. The in-vehicle communication device according to 5. The determination means determines the driver abnormality determined based on the mind and body state of the driver detected by the mind and body state detection means, classified by level according to the degree of abnormality,
When the degree of abnormality of the driver's mind and body condition by the determination unit is low, the control unit is configured so that the video of the driver's face is smaller than the video by the video signal input by the input unit. When the degree of abnormality of the driver's mind and body state by the determination unit is high, the driver's face image is combined so as to be larger than the image by the video signal input by the input unit. The in-vehicle communication device according to claim 1. Voice input means for inputting voices of passengers other than the driver;
Voice output means for outputting the voice input by the voice input means to the driver;
When it is determined by the determination means that the driver's mind and body condition is abnormal, the voice input means and the voice output means are connected, and the voice of the occupant input from the voice input means, The in-vehicle communication device according to claim 1, further comprising: a connection circuit that causes the driver to output via the voice output unit.

Images