JP2004314855A - Apparatus operation control method and apparatus operation control system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus operation control method and a system capable of controlling the operation of a vehicle on-board apparatus by means of the opening and shutting of driver's eyes. <P>SOLUTION: An apparatus control device 2 is connected with a camera device 3 through a first wire harness 10b and connected with the on-board apparatuses 5-8 as objects to be controlled through a network cable 1a. Images concerning the opened/shut condition of the driver's eyes photographed by the camera device 3 are fed to the control device 2, which senses the opened and shut condition of the eyes from the fed images and specifies the apparatus operation corresponding to the sensed contents on the basis of a table stored internally. The control device 2 prepares control data according to the specified contents and transmits them to the on-board apparatus to be controlled, and the operation of the apparatus having received the control data is controlled on the basis of the control data. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an on-vehicle device control method and an on-vehicle device control system that operate on-vehicle devices such as an audio device and a navigation device by changing the state of a face.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, various functions of an in-vehicle device such as an audio device are operated by manually operating various switches, buttons, and the like provided on the device by a driver of a vehicle. When a driver performs a device operation in addition to a driving operation of a vehicle, an operation burden on the driver is large. Therefore, there is a system that can operate the device by voice.
[0003]
The system that performs device operation by voice has a configuration in which a voice operation device having a microphone is connected to the in-vehicle device, the voice emitted by the driver is acquired by the microphone, the obtained voice is recognized by the voice operation device, and the recognized contents The voice operation device controls the operation of the in-vehicle device based on. For example, when the driver emits a voice saying "air conditioning, cooling, on", the voice operation device controls the air conditioner so that the air conditioner mounted on the vehicle blows cool air.
In addition, the device operation by voice is also disclosed in Patent Document 1 below.
[0004]
[Patent Document 1]
JP 2003-15689 A
[0005]
[Problems to be solved by the invention]
In the above-described device operation using voice, since the voice operation device controls the device operation based on the voice acquisition, there is a problem that if the voice cannot be obtained accurately, the device operation cannot be reliably controlled.
[0006]
For example, in a state where a window of a vehicle is opened, external noise may enter the vehicle, and it may be difficult to obtain an accurate sound in the vehicle. Further, even when the window is closed, it may be difficult to obtain an accurate sound due to an output sound from a speaker of the audio device and a sound generated inside the vehicle such as a conversation between occupants of the vehicle. In such a situation, control of the required device operation by voice is hindered.
[0007]
The present invention has been made in view of such circumstances, and by controlling the operation of the on-vehicle device based on the image of the driver's face state, the on-board device can be reliably mounted regardless of the opening / closing of the window and the situation in the vehicle. It is an object of the present invention to provide a device operation control method and a device operation control system that can control the operation of a device.
In addition, the present invention provides a device operation control method and a device operation control method that improve the reliability of processing related to operation control and reduce the processing load by using a table in which a face state and an operation of an on-vehicle device are associated. The purpose is to provide a system.
[0008]
Furthermore, the present invention provides a device operation control system that performs a reliable control of device operation by comparing a face state with a basic image and detecting a time related to detection of the face state. Aim.
Further, the present invention provides a device operation control system which starts control of device operation according to the facial state by judging a face state for control start and further receiving a device control start instruction. The purpose is to do.
[0009]
Further, the present invention provides a device operation control system in which control contents can be easily confirmed by outputting a control content by an audio output device, and further, by displaying contents relating to the device operation on a display device. The purpose is to provide.
Still another object of the present invention is to provide a device operation control system that detects an open / closed state of an eye as a face state so that an operation related to a device operation can be performed by opening / closing an eye.
[0010]
[Means for Solving the Problems]
An apparatus operation control method according to a first invention is an apparatus operation control method for controlling an operation of an in-vehicle apparatus, the apparatus being for imaging a face of a vehicle occupant, outputting an image taken, an imaging apparatus, and the in-vehicle apparatus. A device control device connected to a device, the device control device detects a face state related to an image output from the imaging device, and for controlling the operation of the vehicle-mounted device based on the detected face state. Control data is generated, and the generated control data is output to the in-vehicle device. The operation of the in-vehicle device is controlled by control data output from the device control device.
[0011]
In the device operation control method according to a second aspect, the device control device includes a table in which the face state and the operation of the vehicle-mounted device are associated with each other, and specifies the operation corresponding to the face state detected based on the table. It is characterized in that control data is generated according to the contents of the control.
[0012]
An apparatus operation control system according to a third aspect of the present invention is an apparatus operation control system for controlling the operation of an in-vehicle apparatus, which is for imaging a face of a vehicle occupant, outputs an image taken, an imaging apparatus, and the in-vehicle apparatus. A device control device connected to a device, wherein the device control device detects a face state related to an image output from the imaging device, and a face state detected by the face detection unit. Data generating means for generating control data for operation control of the in-vehicle device based on the data, and output means for outputting the control data generated by the data generating device to the in-vehicle device; The operation is controlled by the control data output from the CPU.
[0013]
According to a fourth aspect of the present invention, in the device operation control system, the device control device further includes: a table in which the face state and the operation of the on-vehicle device are associated; and an operation corresponding to the face state detected by the face detection unit based on the table. And a data generation unit that generates control data corresponding to the content specified by the specification unit.
[0014]
In a device operation control system according to a fifth aspect, the device control device further includes a unit that previously stores an image output from the imaging device as a basic image, and the face detection unit includes the basic image and the image pickup device. It is characterized in that the face state is detected by comparing the images output from the apparatus.
[0015]
A device operation control system according to a sixth aspect of the present invention is the device control device, wherein the device control device further includes: a measuring unit that measures a time after the face detecting unit detects the face state; a unit that stores a reference time; And comparing means for comparing the time measured by the measuring means, wherein the data generating means, when the time measured by the measuring means exceeds the reference time, by comparing the comparing means, It is characterized in that it is configured to generate.
[0016]
In the device operation control system according to a seventh aspect, the device control device further includes a unit for storing a control start face state, and a comparison between the control start face state and the face state detected by the face detection unit. Control means for performing control, wherein the data generation means performs control when the face state for control start and the face state detected by the face detection means are equal by comparison of the control start comparison means. It is characterized by comprising means for setting a state for generating data.
[0017]
The device operation control system according to an eighth aspect of the present invention is further provided with a receiving unit connected to the device control device and receiving a device control start instruction, wherein the data generation unit receives the start instruction. , Means for setting a state in which control data is generated.
A device operation control system according to a ninth aspect further comprises an audio output device connected to the device control device, wherein the device control device further outputs the content of the control data generated by the data generation unit to the audio output device. And a means for outputting the output contents as a sound.
[0018]
The device operation control system according to a tenth aspect further includes a display device connected to the device control device, wherein the device control device includes a unit that outputs the contents related to the table to the display device, The apparatus is characterized by comprising means for displaying the contents of the output table.
An appliance operation control system according to an eleventh aspect of the invention is characterized in that the face detection means is configured to detect an eye open / closed state as a face state.
[0019]
In the first invention and the third invention, the device control device generates control data and outputs the control data to the on-vehicle device to be controlled based on the driver's face state detected from the captured image. The driver can operate the in-vehicle device by appropriately changing the facial condition. Therefore, the driver can operate various in-vehicle devices only by setting a desired face state, not by hand, while driving, and is influenced by the opening / closing state of the vehicle window, the sound generation state in the vehicle, and the like. In-vehicle equipment can be operated without the need.
[0020]
According to the second invention and the fourth invention, since the operation of the vehicle-mounted device is specified based on the table in which the face state and the operation of the vehicle-mounted device are associated, the device control device that determines the operation according to the detected face state The efficiency of the processing and the reduction of the load can be achieved, and the operation control of the in-vehicle device can be quickly realized.
[0021]
According to the fifth aspect, since the face state is detected by comparing the face state of the captured image with the reference image stored in advance, the accuracy of the detection of the face state can be improved. The reliability of operation control of in-vehicle devices can be improved.
According to the sixth aspect of the present invention, the time after the detection of the facial condition is measured, and when the measured time exceeds the reference time, control data is generated to control the device operation. One of the facial states can be reliably detected, and the operability of the device can be improved by distinguishing the unintended facial state from the facial state intended to operate the device.
[0022]
According to the seventh invention, the control start face state is stored, and the stored state is compared with the detected face state. When both are equal, the control of the device operation is started and the face state is determined. Since the mode is shifted to the operation mode, it is possible to prevent a situation in which the operation of the vehicle-mounted device is performed without permission even if the face state is changed carelessly. Note that, in the transition to the operation mode based on the facial state, it is preferable to store complicated contents as the facial state for control start from the viewpoint of preventing unintended transition of the operation mode.
[0023]
According to the eighth aspect, since the receiving means for receiving the instruction to start the device control is provided, the shift to the operation mode due to the change in the face state can be reliably performed by the instruction operation by the receiving means. Note that a switch that can be switched on and off can be applied as the reception unit, and it is preferable that the switch to the on state shifts to the operation mode based on the face state.
[0024]
According to the ninth aspect of the present invention, since the audio output device for outputting the content of the control data by voice is provided, the control content due to the change of the face state can be confirmed by voice, and it is confirmed whether the in-vehicle device is operated as intended. Nature can be secured.
According to the tenth aspect, a table in which the change of the face state and the operation of the in-vehicle device are associated is displayed on the display device, so how the face state is changed and the in-vehicle device operates. Can be determined at a glance by the driver and the occupant of the vehicle, and the usability of the device operation control system can be greatly improved.
[0025]
According to the eleventh aspect, since the open / closed state of the eyes is detected as the face state, the in-vehicle device can be operated by opening and closing the eyes that are most likely to change in the face, and the operability of the in-vehicle device can be improved. In addition, by opening and closing the left and right eyes separately, a plurality of types of states can be easily created. By combining these states, it is possible to control various operations of the vehicle-mounted device.
[0026]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, the present invention will be described with reference to the drawings showing the embodiments.
FIG. 1 shows an overall configuration of a device operation control system 1 according to an embodiment of the present invention. The device operation control system 1 is a device control device 2 in which a camera device 3 as an image pickup device and a speaker 4 as an audio output device are connected by first and second wire harnesses 10a and 10b. 5 to 9 so that the device control device 2 transmits control data through the network cable 1a.
[0027]
The device operation control system 1 of the present embodiment can control the operation of the in-vehicle device by setting the left and right eyes appropriately opened and closed as the face state by the driver. The on-vehicle devices to be controlled correspond to the DVD device 5, the CD reproducing device 6, the MD reproducing device 7, the navigation device 8, and the like connected by the network cable 1a.
[0028]
The camera device 3 is attached to the boss Sc of the steering wheel Sb of the vehicle S as shown in FIG. 2 so as to capture an image of the driver's face. The camera device 3 also performs a process of outputting the captured image to the device control device 2 through the first wire harness 10a. Since the camera device 3 rotates together with the steering wheel Sb, the camera device 3 tilts and rotates according to the operation of the steering wheel Sb.
[0029]
FIG. 3 is a block diagram illustrating an internal configuration of the device control device 2. The device control device 2 includes a speaker connection portion 12 for connection to the speaker 4, a camera connection portion 11 for connection to the camera device 3, a CPU 17 for overall control of the device control device 2, and an image related to device control. An image processing ASIC (Application Specific IC) 13 which is a custom IC dedicated to processing and the like, a storage unit 16 for storing a table to be described later and images related to imaging, a network interface unit 15 for connection to the network cable 1a, And it has the operation part 14 which receives operation with respect to a system.
[0030]
The speaker connection unit 12 is connected to the second wire harness 10b, and performs a process of outputting the content of the control data generated by the image processing ASIC 13 as audio data, as described later. The camera connection unit 11 is connected to the first wire harness 10a, and performs a process of receiving an image output from the camera device 3 and transmitting the image to the image processing ASIC 13.
[0031]
The storage unit 16 functions as a unit that stores the table 20 illustrated in FIG. 4, a basic image related to the driver's eye open / closed state, a reference time related to control data generation, and an eye open / closed state for control start. ing.
[0032]
The table 20 shown in FIG. 4 defines the contents corresponding to the open / closed state of the eyes and the operation of each of the in-vehicle devices 5 to 8. For example, for the MD reproducing device 7, the operation of reproducing the next music by closing the right eye once by the driver is performed. Hereinafter, by closing the right eye twice, the last music is reproduced and the left eye is reproduced. It is stipulated that closing the screen once reproduces the previous music, closing the left eye twice reproduces the first music, and closing both eyes stops the reproduction.
[0033]
When changing the in-vehicle device to be controlled, the in-vehicle device to be controlled in the table 20 is lowered by one step by closing the left eye once and then closing the right eye once. When the in-vehicle device is the MD playback device 7, it is defined that the control target is changed to the CD playback device 6 by closing the left eye once and closing the right eye once.
[0034]
Further, it is defined that the on-vehicle device to be controlled moves up one step in the table 20 by closing the right eye once and then closing the left eye once. For example, when the current control target is the navigation device 8, it is defined that the control target is changed to the DVD device 5 by closing the right eye once and then closing the left eye once.
[0035]
The storage unit 16 in FIG. 3 stores images 21a to 21d captured by the camera device 3 illustrated in FIGS. 5A to 5D as basic images in advance. An image 21a in FIG. 5A is an image in which the driver has both eyes open, and an image 21b in FIG. 5B is an image in which the right eye is closed, and FIG. 5C. The image 21c of FIG. 5 is an image of the state where the left eye is closed, and the image 21d of FIG. 5D is an image of the state where both eyes are closed. By storing the basic image in advance in this way, the open / closed state of the eyes related to the image captured by the camera device 3 can be detected by comparison with the basic image. Faster processing has been achieved.
[0036]
Further, the images 21a to 21d are controlled by the image processing ASIC 13 by operating the reference image switch provided on the operation unit 14 in association with the eye states such as the binocular opening, the right eye closing, the left eye closing, and the binocular closing. It is stored in the storage unit 16.
[0037]
The storage unit 16 stores a time of one second as a reference time for control data generation. Further, the storage unit 16 stores contents such as right-eye closed, left-eye closed, right-eye closed, and left-eye closed as the open / closed state of the eyes for control start. I remember the time.
[0038]
The operation unit 14 includes the above-described reference image switch, control start switch, and the like. The on / off control start switch corresponds to reception of a device control start instruction, and when turned on by operation, activates a function related to the face detection unit of the image processing ASIC 13 to open and close the eyes. To shift to an operation mode in which each in-vehicle device can be operated. By providing the control start switch in the operation unit 14 in this way, even if the user opens and closes his eyes in a situation where the operation of the device is not intended, the in-vehicle device does not operate without permission.
[0039]
The image processing ASIC 13 functions as a face detection unit, a control start comparison unit, a specification unit, a data generation unit, a measurement unit, a comparison unit, an output unit, and the like. The face detection means performs a process of detecting the open / closed state of the eyes on the driver's face in the image captured by the camera device 3 and received by the camera connection unit 11.
[0040]
The face detection unit of the present embodiment compares the images 21 a to 21 d of FIGS. 5A to 5D stored in the storage unit 16 with the images received by the camera connection unit 11 to obtain the received image. It is determined which of the four types the driver's eye opening / closing state corresponds to. When the steering wheel Sb is operated and the camera device 3 is rotated and tilted, it is difficult to compare the tilt of the driver image shown in the image, and therefore, the device operation due to the open / closed state of the eyes is stopped. .
[0041]
The control start comparing means has a function equivalent to that of the control start switch of the operation unit 14, and includes the open / closed state of the eyes detected by the face detection means and the open / close state of the control start eyes stored in the storage unit 16. The contents of the state are compared, and if both states are equivalent, the operation mode is shifted to the operation mode by opening and closing the eyes. That is, as described above, according to the opening / closing state of the control start eye stored in the storage unit 16, when the driver opens and closes the eyes within three seconds, such as closing the right eye, closing the left eye, closing the right eye, and closing the left eye. The mode is shifted to the operation mode.
[0042]
Therefore, in the device operation control system 1, the operation mode is changed to the operation mode by opening and closing the eyes by turning on the control start switch of the operation unit 14 or performing the above-described open / close state of the eyes for control start, and inadvertently mounting the vehicle. The device is prevented from operating.
[0043]
The specifying means of the image processing ASIC 13 specifies the operation of the vehicle-mounted device corresponding to the eye open / closed state detected by the face detecting means based on the table 20 shown in FIG. In addition, the specifying unit is configured to specify the operation of the on-board device to be controlled with respect to the MD playback device 7 immediately after the device operation control system 1 shifts to the operation mode by opening and closing the eyes. Note that the image processing ASIC 13 specifies the device operation based on the table 20, thereby reducing the processing load related to the operation specification and improving the processing efficiency.
[0044]
Further, the data generation unit of the image processing ASIC 13 generates control data for operation control according to the operation content of the vehicle-mounted device identified by the identification unit based on the open / closed state of the eyes detected by the face detection unit. For example, immediately after the device operation control system 1 shifts to the operation mode by opening and closing the eyes, when the driver closes the right eye once, the data generation unit instructs the MD reproduction device 7 to reproduce the next music. A packet including a command of the content is generated as control data, and when the driver closes the left eye once and the right eye continuously, it is determined that the CD playback device 6 is to be controlled.
[0045]
In the operation mode corresponding to a state in which control data is generated, the data generation unit performs processing in which the image processing ASIC 13 functions as a measurement unit and a comparison unit in order to reliably operate the vehicle-mounted device by opening and closing the eyes. The control data is generated using the results. The measuring means of the image processing ASIC 13 measures the time after the face detecting means has performed the detecting process, and the comparing means compares the time measured by the measuring means with one second of the reference time stored in the storage unit 16. Is what you do.
[0046]
The data generation means generates control data when the time measured by the comparison means exceeds one second, and by adopting such a configuration, the opening / closing state of the eyes for each operation is determined. The control data is generated by reliably determining. In the present embodiment, the eye opening and closing corresponding to the device operation is performed within one second, and it is required that the state of closing a plurality of eyes such as closing the right eye twice is also performed within one second.
[0047]
Therefore, when one second elapses after closing the right eye once, control data of the device operation for the state where the right eye is closed once is generated, and when one second elapses after closing the right eye twice within one second, the right eye The control data of the device operation with respect to the state where is closed twice is generated. Furthermore, if the right eye is closed twice within one second after the right eye is closed once, the state where the right eye is closed three times is not specified in the table 20, so that the first right eye is closed once. It will be ignored. However, if one second elapses after closing the right eye twice, control data for closing the right eye twice is generated.
[0048]
In the present embodiment, since the device operation is performed based on the contents related to the above-described eye opening / closing, after the eye opening / closing for the device operation, if the eye opening / closing is not performed within one second, the device operation is determined. It is important to notify the driver in advance of this fact from the viewpoint of performing a reliable operation. Note that the above-described rules regarding the opening and closing of the eyes are also applied to a case where the control target is changed.
[0049]
In addition, the image processing ASIC 13 transmits, as an output unit, a packet of the generated control data to the network interface unit 15 and transmits the packet from the network interface unit 15 through the network cable 1a.
[0050]
Further, the image processing ASIC 13 performs a process of outputting the contents of the generated control data to the speaker 4 from the speaker connection unit 12 as audio data. For example, when control data for reproducing the next music is generated by the MD reproduction device 7, the image processing ASIC 13 generates and outputs audio data "MD reproduction device, reproduction of next music". When such voice data is output, the speaker 4 performs a process of outputting the content of the voice data by voice, so that the driver can check the content of operation of the vehicle-mounted device by opening and closing eyes.
[0051]
Further, the image processing ASIC 13 also performs a process of transmitting data related to the table 20 stored in the storage unit 16 from the network interface unit 15 to the monitor device 9 corresponding to a display device. When transmitted in this manner, the monitor device 9 performs a process of displaying a table 20 as shown in FIG. 4 on the screen. By displaying the table 20 in this manner, the driver can check the operation content of the on-vehicle device corresponding to the opening / closing of the eyes on the screen of the monitor device 9, and can perform the eye operation according to the operation content desired to be operated. There is no need to store the open / closed state.
[0052]
Note that, when the in-vehicle device is actually operating by opening and closing the eyes, the image processing ASIC 13 generates and transmits data in which the column of the operation content in the table 20 is displayed in reverse video with a difference from the others. By doing so, the current operation status can be understood at a glance by the screen display of the monitor device 9. For example, when the MD playback device is playing the next song, the columns of “MD playback device” and “next song” are highlighted. In addition, by displaying the screen in this manner, even the occupants of the vehicle S other than the driver can confirm the current operation status of the in-vehicle device and the operation content by opening and closing the eyes.
[0053]
Further, when each of the in-vehicle devices 5 to 8 connected to the network cable 1a shown in FIG. 1 receives the packet addressed to itself transmitted from the device control device 2 as described above, the operation is performed by the control data relating to the packet. It is configured to be controlled. With the above-described configuration, the device operation control system 1 enables the operation of each of the in-vehicle devices 5 to 8 by opening and closing the eyes of the driver.
[0054]
Next, in the device operation control system 1 having the above-described configuration, a series of processing procedures according to a device operation control method in a case where a control start switch of the operation unit 14 is turned on to shift to an operation mode based on an eye open / closed state. Will be described based on the flowchart of FIG.
First, it is determined whether the control start switch provided on the operation unit 14 of the device control device 2 has been turned on by the driver (S1). If the control start switch has not been turned on (S1: NO) ), And waits for it to be turned on. Further, when the control start switch is turned on (S1: YES), the system is activated and shifts to the operation mode, and the opening / closing state of the driver's eyes is imaged by the camera device 3 (S2). The image is output to the device control device 2.
[0055]
The device control device 2 detects the open / closed state of the eyes related to the output image (S3), and starts measuring the time elapsed after the detection (S4). Note that the detected eye open / closed state in this flowchart means the eye open / closed state corresponding to one device operation.
[0056]
Next, the device control device 2 determines whether or not the measured time has exceeded the reference time stored in the storage unit 16 (S5). If the measured time has not exceeded the reference time (S5: NO), the device control device 2 performs device control. The apparatus 2 determines whether or not the detected open / closed state of the eyes has changed (S6). If there is no change in the eye open / closed state (S6: NO), the device control device 2 continues the time measurement (S7), and returns to the step (S5) as to whether or not the reference time has been exceeded. When there is a change in the eye open / closed state (S6: YES), the process returns to the step of starting the time measurement (S4), and it is determined whether or not the reference time has been exceeded after the change (S5). ).
[0057]
On the other hand, when the measured time exceeds the reference time (S5: YES), the device control device 2 specifies the operation of the on-vehicle device corresponding to the detected eye condition based on the table 20 (S8), and sets the specified content. The corresponding control data is generated (S9), and the control data is transmitted to the specified in-vehicle device (S10). A predetermined operation of the in-vehicle device to which the control data is transmitted is controlled by the control data (S11). Note that the device control device 2 outputs the content of the generated control data from the speaker 4 by voice as the control data is generated, so that the driver can confirm the operation content by opening and closing the eyes by voice. Since the table contents operated on the device 9 are displayed on the screen, the operation contents can be confirmed by looking at the screen.
[0058]
As described above, the device operation control system 1 can control the operation of the in-vehicle device by opening and closing the eyes, so that various in-vehicle devices can be reliably operated without being affected by the sound in the vehicle and without performing manual operation. ing.
[0059]
The device operation control system 1 according to the present invention is not limited to the above-described embodiment, and various modifications can be applied. For example, as the on-vehicle device to be controlled, other than the on-vehicle devices 5 to 8 shown in FIG. 1 can be applied. The content is not limited to the content, and another correspondence may be defined.
[0060]
When the configuration is simplified, the device operation control system 1 may be configured to omit the processing part for outputting the content of the control data of the device control device 2 and the speaker 4. The processing function of transmitting the data to the monitor device 9 and displaying the data on the screen may be omitted. Further, the device operation control system 1 may be configured so that the control start switch of the operation unit 14 for shifting to the operation mode by opening and closing the eyes is omitted, or if the control start switch is left, the control is started. The specification for omitting the process of determining the start based on the specific eye open / closed state may be adopted.
[0061]
In addition to the attachment to the steering wheel Sb, the camera device 3 may be attached to the upper surface of the instrument panel, the upper edge of the windshield, or the like as long as it can capture the driver's face. Further, in order to control the device operation, it is also possible to use the opening and closing of the mouth, the change of the facial expression, and the like, in addition to the opening and closing of the eyes.
[0062]
【The invention's effect】
As described in detail above, in the first invention and the third invention, the device control device generates control data and outputs the control data to the on-vehicle device to be controlled based on the image of the driver's face. Therefore, the driver can operate the vehicle-mounted device by appropriately changing the facial condition.
In the second invention and the fourth invention, since the operation of the vehicle-mounted device is specified based on the table in which the face state and the operation of the vehicle-mounted device are associated, the processing of the device control device is efficiently performed, and the vehicle-mounted device is quickly performed. Operation can be controlled.
[0063]
According to the fifth aspect, by comparing the face state of the captured image with the reference image stored in advance, the accuracy of detecting the face state can be improved.
According to the sixth aspect, since the time after detecting the face state is measured and compared with the reference time, one face state can be reliably detected, and the unintended face state and the unintended face state in which the device operation is not intended. Thus, the operation of the vehicle-mounted device can be prevented.
[0064]
According to the seventh aspect, since the control mode is changed to the operation mode based on the face state by comparing the control start face state with the detection-related face state, the operation of the on-vehicle device can be performed even if the face state is inadvertently changed. It is possible to prevent a situation that occurs without permission.
According to the eighth aspect, since the receiving means for receiving the instruction to start the device control is provided, the shift to the operation mode due to the change in the face state can be reliably performed by the instruction operation by the receiving means.
[0065]
According to the ninth aspect, the content of the control data is output by the voice output device, so that the control content can be confirmed by the voice.
According to the tenth aspect, since the contents related to the table in which the change of the face state and the operation of the in-vehicle device are associated are displayed on the display device, the driver and the occupant of the vehicle can use the facial state when operating the in-vehicle device. Can be confirmed on the screen, and the usability of the device operation control system can be improved.
[0066]
According to the eleventh aspect, since the open / closed state of the eyes is detected as the face state, the in-vehicle device can be operated by opening and closing the eyes, which are most likely to change in the face, and the operability of the in-vehicle device can be improved.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram of a device operation control system according to an embodiment of the present invention.
FIG. 2 is a schematic diagram showing an attached state of the camera device.
FIG. 3 is a block diagram showing an internal configuration of the device control device.
FIG. 4 is a table related to a table in which the open / closed state of an eye and the operation of an in-vehicle device are associated with each other.
FIGS. 5A and 5B are schematic diagrams showing basic images related to the open / closed state of various eyes.
FIG. 6 is a flowchart according to a device operation control method.
[Explanation of symbols]
1 Equipment operation control system
2 Equipment control device
3 Camera device
4 Speaker
5 DVD device
6 CD playback device
7 MD playback device
8 Navigation device
9 Monitor device
13 Image Processing ASIC
14 Operation unit
15 Network interface
16 Storage unit
20 tables
S vehicle
Sb steering wheel

Claims (11)

In a device operation control method for controlling the operation of an in-vehicle device,
An imaging device for imaging a face of a vehicle occupant, and outputting the captured image;
Comprising an imaging device and a device control device connected to the in-vehicle device,
The device control device,
Detecting a face state related to the image output from the imaging device,
Generate control data for operation control of the in-vehicle device based on the detected face state,
Output the generated control data to the in-vehicle device,
The in-vehicle device,
An apparatus operation control method, wherein an operation is controlled by control data output from the apparatus control device. The device control device,
Equipped with a table corresponding to the facial state and the operation of the on-board equipment,
Specifying an operation corresponding to the detected face state based on the table,
2. The apparatus operation control method according to claim 1, wherein control data according to the specified content is generated. In an equipment operation control system that controls the operation of in-vehicle equipment,
An imaging device for imaging a face of a vehicle occupant, and outputting the captured image;
Comprising an imaging device and a device control device connected to the in-vehicle device,
The device control device,
Face detection means for detecting a face state related to the image output from the imaging device,
Data generation means for generating control data for operation control of the vehicle-mounted device based on the face state detected by the face detection means,
Output means for outputting the control data generated by the data generation means to the in-vehicle device,
The in-vehicle device,
An apparatus operation control system, wherein an operation is controlled by control data output from the apparatus control device. The device control device further includes:
A table corresponding to the face state and the operation of the in-vehicle device,
Specifying means for specifying an operation corresponding to the face state detected by the face detection means based on the table,
The data generation means includes:
The device operation control system according to claim 3, further comprising a unit configured to generate control data corresponding to the content specified by the specifying unit. The device control device further includes:
Means for pre-storing an image output from the imaging device as a basic image,
The face detection means,
The apparatus operation control system according to claim 3 or 4, wherein a face state is detected by comparing the basic image and an image output from the imaging device. The device control device further includes:
Measuring means for measuring the time since the face detecting means detects the face state,
Means for storing the reference time;
Comparing means for comparing the reference time and the time measured by the measuring means,
The data generation means includes:
The device operation control system according to claim 3, wherein, when the time measured by the measuring means exceeds the reference time, the control data is generated. The device control device further includes:
Means for storing a face state for control start;
Comprising: a control start comparing means for comparing the control start face state and the face state detected by the face detection means,
The data generation means includes:
4. The control device according to claim 3, wherein the control start comparison unit compares the control start face state with the face state detected by the face detection unit. Item 7. An apparatus operation control system according to any one of Items 6. It is connected to the device control device, further comprising a receiving means for receiving a device control start instruction,
The data generation means includes:
The device operation control system according to claim 3, further comprising a unit that sets a state in which control data is generated when the receiving unit receives a start instruction. Further comprising an audio output device connected to the device control device,
The device control device further includes:
Means for outputting the content of the control data generated by the data generating means to the audio output device,
The audio output device,
The device operation control system according to any one of claims 3 to 8, further comprising: means for outputting the output content by voice. Further comprising a display device connected to the device control device,
The device control device,
Means for outputting the contents related to the table to the display device,
The display device,
The device operation control system according to any one of claims 4 to 9, further comprising means for displaying the contents of the output table. The device operation control system according to claim 3, wherein the face detection unit is configured to detect an eye open / closed state as a face state.

Images