JP2012146209A - Electronic device and control program for electronic device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which it is difficult to more reliably cause a device to recognize the mental condition of a user before a change in the user is expressed as biometric information or even when it has been expressed.SOLUTION: In order to solve the problem, an electronic device comprises: an utterance speed detection unit for detecting the utterance speed of a target person; and a control unit for controlling a device to be controlled based on a detection result of the utterance speed detection unit. Then, a control program for the electronic device causes a computer to execute: an utterance speed detection step for detecting the utterance speed of the target person; and a control step for controlling the device to be controlled based on a detection result of the utterance speed detection step.

Description

  The present invention relates to an electronic device and a control program for the electronic device.

In recent years, it has been proposed to obtain user's biometric information and provide various kinds of support, detect whether the user is concentrated on music, and lower the volume when not concentrated on music. Sound reproduction devices have been proposed.
[Prior art documents]
[Patent Literature]
[Patent Document 1] JP-A-2005-34484

  However, it is difficult to make the device recognize the mental state of the user more reliably before the change of the user appears as the biometric information or even when it appears.

  In order to solve the above-described problem, the electronic device according to the first aspect of the present invention controls the controlled device based on the speech rate detector that detects the speech rate of the target person and the detection result of the speech rate detector. A control unit.

  In order to solve the above-described problem, an electronic device control program according to a second aspect of the present invention includes an utterance speed detection step for detecting an utterance speed of a target person, and a control target based on a detection result of the utterance speed detection step And causing the computer to execute control steps for controlling the device.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

It is a figure which shows the outline | summary of the concentration level detection system which concerns on 1st Embodiment. 1 is a block diagram of a concentration level detection system according to a first embodiment. It is a flowchart which shows the process of the concentration level detection system which concerns on 1st Embodiment. It is a flowchart which shows the process regarding the detection of a user's hand as an application example of 1st Embodiment. It is a flowchart which shows the process regarding the detection of a user's speech speed as an application example of 1st Embodiment. It is a figure which shows the outline | summary of the smart phone showing the modification of 1st Embodiment. It is a block diagram of the concentration level detection system which concerns on the modification of 1st Embodiment. It is a figure which shows the outline | summary of the concentration level detection system which concerns on 2nd Embodiment. It is a block diagram of the concentration level detection system which concerns on 2nd Embodiment. It is a flowchart which shows the process of the concentration level detection system which concerns on 2nd Embodiment. It is a display example of a display visually recognized by the presenter. It is a display example of a display visually recognized by the presenter.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 is a diagram showing an overview of a concentration level detection system 110 according to the first embodiment. As shown in FIG. 1, the concentration level detection system 110 includes a personal computer (personal computer) 200 and a biosensor 330 worn by the user. The personal computer 200 includes user input operation members such as a display 201, a keyboard 202, and a touch pad 203. In addition, a mouse 300 is connected to the personal computer 200, and the user can give an instruction to the personal computer 200 by operating the mouse 300.

  The personal computer 200 further includes a built-in camera 204, an ultrasonic sensor 205, a speaker 206, and a microphone 207. The built-in camera 204 includes a photographing lens and an image sensor. An image sensor such as a CCD sensor or a CMOS sensor is used as the image sensor. The built-in camera 204 is disposed on the upper part of the display 201 and has an angle of view capable of simultaneously photographing the operation members such as the keyboard 202 and the touch pad 203 together with the upper body including the user's face, hands, and arms. Instead of the built-in camera 204, for example, a camera module that can be mounted in the vicinity of the display 201 by a clip may be adopted. The ultrasonic sensor 205 is provided in the vicinity of the built-in camera 204, and executes transmission / reception of ultrasonic waves for measuring the distance from the display 201 to the user.

  Corresponding to the left and right palm rest positions in the vicinity of the keyboard 202, a temperature control unit 208 is provided in the personal computer 200. The temperature control unit 208 has a heating wire such as a nichrome wire or an iron chrome wire, for example, and generates a temperature rise when an electric current is applied. The user can feel a temperature change through the palm.

  Piezoelectric sensors 209 are provided on the back surface of the keyboard 202 corresponding to the individual keys. The piezoelectric sensor 209 includes a piezo element, and electrically detects vibration by converting an externally applied force into a voltage by a piezoelectric effect. Accordingly, the piezoelectric sensor 209 can detect the strength with which the user strikes the key and the repeated operation.

  A floor sensor 310 is provided at the user's foot. The floor sensor 310 is configured by a piezo element or the like in the same manner as the piezoelectric sensor 209, and detects the motion of the foot such as the user's stepping or poor sliding. The floor sensor 310 is connected to the personal computer 200 and transmits the detected signal to the personal computer 200.

  A ceiling camera 320 is provided in a ceiling portion near the user's head. The ceiling camera 320 includes a photographic lens and an image sensor, and is adjusted to an angle of view capable of photographing the user's head. The ceiling camera 320 transmits the captured image signal to the personal computer 200 via a wireless LAN, for example. On the other hand, the personal computer 200 transmits control signals such as a start of photographing and a request for a photographed image signal to the ceiling camera 320.

  The biosensor 330 is mounted, for example, wrapped around the user's arm. The biological sensor 330 senses the user's biological information and transmits the output to the personal computer 200. A specific configuration will be described later.

  The personal computer 200 is connected to a telephone 400 as a bidirectional communication device. The telephone 400 receives a control signal from the personal computer, and restricts or cancels its function. In addition, the indicator light 410 is connected to the personal computer 200. The indicator lamp 410 includes, for example, a high-intensity LED that can change the emission color. The indicator light 410 makes the surrounding people recognize the user's concentration state determined by the personal computer 200 by emitting light in red, for example.

  FIG. 2 is a block diagram of the concentration level detection system according to the first embodiment. As shown in the figure, a personal computer 200 includes elements such as a display 201 and a keyboard 202 described with reference to FIG. 1 with a personal computer CPU 210 that performs overall control as a center.

  The timer 211 starts timing in response to a start instruction from the personal computer CPU 210 and returns the time to the personal computer CPU 210 in response to an end instruction. The ROM 212 is a non-volatile memory such as a flash memory, for example, and plays a role of storing a program for controlling the personal computer 200, various parameters, and the like. Also, various data, user schedules, usage status of the personal computer 200, biometric information data, output of the floor sensor 310, and the like can be stored.

  The emotion analysis unit 213 receives the biological information from the biological sensor 330 and analyzes the user's emotion. The biological sensor 330 is a sensor that detects the biological information of the user. For example, the biological sensor 330 detects the pulse by irradiating the living body with the LED toward the living body and receiving the light reflected from the living body with respect to the irradiation light. Including a pulse sensor. The configuration is disclosed in, for example, Japanese Patent Laid-Open No. 2005-270543 (US Pat. No. 7,538,890). In addition to the pulse sensor, the biological sensor 330 can also detect a user's sweating amount by providing a sweating sensor provided with a plurality of electrodes. Furthermore, a temperature sensor that measures body temperature and a blood pressure sensor that measures blood pressure can be provided.

  The emotion analysis unit 213 receives the biological information from the biological sensor 330 and determines the user's emotion. For example, when a high heart rate and mental sweating are detected, it can be determined that the user is feeling “impressed”. The correspondence between the output of the biosensor 330 and the emotion is obtained in a verification manner, and a table indicating the correspondence can be stored in the ROM 212. Therefore, the emotion analysis unit 213 determines an emotion depending on whether the acquired biological information matches a specific emotion pattern described in the table. The biosensor 330 is not limited to a wristwatch type that is wrapped around the user's arm, and various forms can be adopted as long as the biosensor 330 comes into contact with a part of the body such as the user's hand or finger (ring-type biosensor). The emotion analysis unit 213 may analyze emotions in consideration of the detection results of the piezoelectric sensor 209 and the floor sensor 310 in addition to the biological information from the biological sensor 330. The emotion analysis unit 214 and image analysis described later may be used. The emotion may be analyzed in consideration of the analysis result of the unit 215.

  The voice analysis unit 214 analyzes the voice captured from the microphone 207. The voice analysis unit 214 has a voice recognition dictionary, and can convert the identified voice into text data and display it on the display 201. Some recent personal computers have voice recognition software installed. Such installed software may be used, or commercially available software may be installed separately.

  In addition, the voice analysis unit 214 cooperates with the personal computer CPU 210 to perform a conversation on the phone 400, a conversation with a colleague in the surroundings, a conversation speed of the user (speech speed), a loudness, a conversation Detect time etc. The speech rate is detected as, for example, the number of output phonemes per unit time or the number of mora per unit time. A mora is a segmental unit of sound with a certain length of time.

  The image analysis unit 215 analyzes the captured image signal captured by the built-in camera 204 and the captured image signal captured by the ceiling camera 320. The image analysis unit 215 performs user face recognition and facial expression recognition. For example, the image analysis unit 215 detects a facial expression with a wrinkle between eyebrows and a facial expression with narrowed eyes rather than a smile from the user's face area in the image signal. In addition, the image analysis unit 215 acquires the timing information of the timer 211 and detects, for example, how long a facial expression with a eyelid between the eyebrows has continued. In addition, when detecting a facial expression with a narrowed eye, the image analysis unit 215 reads out information about the average eye size on the image from the built-in camera 204 from the ROM 212, and the eye that was actually photographed. Detect by comparing with size. The detection of eyebrows between eyebrows may be detected by pattern matching by storing an image with eyebrows between eyebrows in the ROM 212 as a reference image, or may be detected from the shadow distribution of the portion between the left eye and the right eye. . In addition, the detection of the eyelid between eyebrows is also disclosed in, for example, US Patent Publication No. 2008-292148.

  The size of the image captured by the built-in camera 204 depends on the distance between the built-in camera 204 and the user. In the present embodiment, the distance dependence is eliminated by detecting the distance between the built-in camera 204 and the user by the ultrasonic sensor 205 and correcting the size of the image. The distance measurement is not limited to the case where the ultrasonic sensor 205 is provided. For example, a laser distance sensor, an infrared sensor, or the like may be used. Further, if the size of the specific part of the user (for example, the size of the face) is known, the distance between the built-in camera 204 and the user can be calculated by matching the size of the known face with the size of the photographed face.

  Further, the image analysis unit 215 captures the image signal of the ceiling camera 320 and detects the position of the user's head, the amount of movement, and the like. For example, if the image analysis unit 215 detects that the user's head constantly shakes, the personal computer CPU 210 can determine whether the user is distracting or falling asleep. If the position and movement amount of the user's head can be detected by the ultrasonic sensor 205 or the like, the ceiling camera 320 may be omitted. Conversely, if the ceiling camera 320 can detect the distance between the built-in camera 204 and the user, the ultrasonic sensor 205 may be omitted.

  The external connection interface 216 is an interface for connecting to an external device. As the interface, various connection standards such as wireless / wired LAN, USB, HDMI, Bluetooth (registered trademark) can be adopted. For example, the telephone 400 is connected to the personal computer 200 via the external connection interface 216, and the personal computer CPU 210 controls the call rejection when the user's concentration exceeds a predetermined threshold, as will be described later. A signal is transmitted to the phone 400. At this time, the personal computer CPU 210 transmits in parallel to the indicator lamp 410 a control signal for performing light emission indicating a concentrated state. When the concentration state continues for a certain period of time, the personal computer CPU 210 gives a temperature increase command to, for example, a temperature control unit built in the mouse 300.

  FIG. 3 is a flowchart showing processing of the concentration level detection system according to the first embodiment. In this flow, detection of the user's concentration level, facial expression, etc. under the control of the personal computer CPU 210 and processing according to the detection result are executed. In this flow, it is assumed that the user is operating the personal computer 200.

  Personal computer CPU210 inputs the information regarding a user's living body (Step S101). Specifically, the personal computer CPU 210 detects biological information such as the user's pulse, body temperature, and sweat detected by the biological sensor 330, the strength and speed of the user's keyboard 202 detected by the piezoelectric sensor 209, the user's poor shaking, and the user's speech speed. And the amount of speech. As described above, the information related to the living body here is not limited to the information acquired from the living body sensor 330. Further, the personal computer CPU 210 does not need to input all the biological information, and only needs to input biological information that can detect the degree of concentration.

  The personal computer CPU 210 stores the input biometric information of the user in the ROM 212 and records a log of the biometric information of the user. The personal computer CPU 210 detects the concentration level of the user based on the biological information stored in the ROM 212 as described later. In addition, the case where the personal computer 200 is shared and used by a plurality of users can be considered. In such a case, the personal computer CPU 210 recognizes the face of the user with the built-in camera 204 and records a log of biometric information for each user.

  Next, the personal computer CPU 210 performs photographing by the built-in camera 204 and the ceiling camera 320, and facial expression detection by the image analysis unit 215 (step S102).

  The personal computer CPU 210 analyzes whether the eyebrows are between the eyebrows of the user or whether the eyes are narrowed. If it is such a facial expression, it is estimated that the display on the display 201 is difficult to see. The personal computer CPU 210 detects the distance from the display 201 to the user using the ultrasonic sensor 205 when the user is narrowing his eyes. The personal computer CPU 210 determines that the degree of concentration is not high when the eyes are narrowed, and that the degree of concentration is high when there is little change in facial expressions other than the narrowed eyes. In the case of performing this analysis, in addition to the analysis of facial expressions, analysis accuracy can be improved by taking into account the user's emotions (impressed, frustrated, etc.) from the biological information acquired in step S101. it can.

  Further, the image analysis unit 215 detects the amount of movement of the user's head from the image signal of the ceiling camera 320. When the user is concentrated, the amount of movement of the head is small, whereas when the user is not concentrated, the amount of movement of the head is increased. Note that the order of step S101 and step S102 may be interchanged.

  The personal computer CPU 210 proceeds to step S103, and detects the degree of user concentration using the results of steps S101 and S102. In general, when a person is concentrated, the pulse rate and body temperature rise. In addition, when working in a hurry (that is, when the degree of concentration is high), the strength and speed at which the keyboard 202 is struck may be increased, or poor shaking may be performed. In addition, when talking on the phone 400 or the like, there is a case where the voice becomes loud or the voice becomes loud. Furthermore, while the head is concentrated, the head hardly moves. When the head is not concentrated, the head moves more by looking away or depending on the situation. Therefore, in the concentration level detection system 110 of this embodiment, the personal computer CPU 210 detects the user concentration level by comparing the biometric information of the user stored in the ROM 212 with the biometric information input in step S101. In this case, the personal computer CPU 210 may detect the degree of concentration of the user by comparing the biometric information when the user has concentrated in the past with the biometric information input in step S101. Alternatively, it may be determined that the user is concentrated when the pulse and the strength of hitting the keyboard 202 increase by 10% or more.

  It should be noted that the poverty shaking may be performed when concentrated or when not concentrated. In such a case, the personal computer CPU 210 may determine whether the user is a type that shakes poorly when concentrating from other biological information for each user, and may use it for the determination of the subsequent degree of concentration.

  The personal computer CPU 210 proceeds to step S104, and determines whether or not the user concentration acquired in step S103 exceeds a predetermined threshold value. In the present embodiment, the threshold value is set for each user, and is set from the user concentration data stored in the ROM 212. For example, the personal computer CPU 210 sets 10% as a threshold value for the biological information representing the average degree of concentration in the past normal time as described above. As another example, the personal computer CPU 210 may set the concentration level in a situation in which biometric information indicating an irritated emotion is detected as a threshold when a telephone is ringed or spoken when the user has concentrated in the past. it can. Note that the personal computer CPU 210 uses the emotion analysis unit 213 to determine the feeling of frustration based on whether the values such as heart rate and blood pressure match the frustrating emotion pattern described in the table as described above. .

  The personal computer CPU 210 proceeds to step S114 when determining that the user's concentration degree is average, and proceeds to step S105 when determining that the user's concentration degree is high. First, a case where the concentration level does not exceed the threshold will be described.

  The personal computer CPU 210 proceeds to step S114 and determines whether or not the user is frustrated using the results of steps S101 and S102. Note that the personal computer CPU 210 uses the emotion analysis unit 213 to determine the feeling of frustration based on whether the values such as heart rate and blood pressure match the frustrating emotion pattern described in the table. Further, the personal computer CPU 210 uses the user's facial expression analyzed by the image analysis unit 215, the user's poverty detection detected by the floor sensor 310, the utterance volume detected by the voice analysis unit 214, etc., for determination more accurately. You can also In particular, the user's facial expression is determined to be frustrating when it is determined that the user's facial expression is displeased, such as putting a eyelid between the eyebrows or narrowing his eyes instead of a smile, or if the utterance volume is high To do.

  The personal computer CPU 210 proceeds to step S115 when it is determined that the user is frustrated, and returns to step S101 when it is determined that the user is not frustrated. In step S115, the personal computer CPU 210 executes various adjustments.

  Specifically, the personal computer CPU 210 changes the setting of the reaction speed for the input to the keyboard 202. Here, it is estimated that the feeling of frustration is caused by the slow reaction speed of the keyboard 202, and the setting is changed so that the reaction speed becomes faster. Further, when the personal computer CPU 210 detects that the user is narrowing his eyes, the personal computer CPU 210 estimates that it is caused by the small display, and sets the size of characters, figures, icons, etc. displayed on the display 201. Change to a setting that displays larger.

  Here, when increasing the reaction speed of the keyboard 202, the personal computer CPU 210 sets the response speed (sensitivity) of the touch pad 203 using software when the user is performing an operation using the keyboard 202. May be. Such a setting can prevent a malfunction when the user's hand and its vicinity accidentally approach the touch pad 203.

  The personal computer CPU 210 may change the size setting according to the detection result of the ultrasonic sensor 205 when the display 201 is enlarged. The user may be frustrated even when the keyboard 202 is not operated. At this time, when the display 201 is not intended to be changed to the energy saving mode or the screen saver, the user is more frustrated. Therefore, when the user is frustrated, the personal computer CPU 210 changes the setting to make the energy saving mode, the time to shift to the screen saver longer, or prohibit the shift. It should be noted that the adjustment in step S115 need not be performed all of the above, but may be selected and executed as appropriate. Note that the determination accuracy when the user is frustrated or pondering can be improved by using a plurality of analysis units such as the emotion analysis unit 213 and the image analysis unit 215.

  Subsequently, the personal computer CPU 210 proceeds to step S116, and determines whether there is a specific repetitive operation or a continuous operation as an operation state with the keyboard 202. When the same key such as “Back Space” key and “Delete” key is repeatedly operated, or when the key is operated continuously even if the key is not regular, the personal computer CPU 210 performs input from the keyboard 202 Judge that the operation is not good. If the personal computer CPU 210 determines that the input operation is not successful, the process proceeds to step S117. If not, the process returns to step S101.

  When proceeding to step S117, the personal computer CPU 210 changes the operation setting from the keyboard 202 to the voice input using the microphone 207 from the user 202. Then, the personal computer CPU 210 displays on the display 201 that the operation setting has been changed. Of course, before changing the operation setting, the personal computer CPU 210 may obtain an understanding to the user that the operation setting is to be changed.

  Alternatively, when it is determined that the repetitive operation and the continuous operation are in the order of the character conversion candidates in the kanji conversion, for example, the description of the definition file that defines the order is rewritten and the order displayed on the display 201 is changed. . Further, the personal computer CPU 210 may change the conversion input from the Roman character input to the hiragana input, or may change the initial setting of the alphanumeric characters from the upper case to the lower case. Furthermore, the personal computer CPU 210 can invalidate the learning effect during the period of repeated operation and continuous operation.

  Using the emotion analysis unit 213, the personal computer CPU 210 proceeds to step S118, and determines whether or not the user's frustration continues. If the personal computer CPU 210 determines that the feeling of irritation continues, the process proceeds to step S119, and if it is determined that the feeling of irritation has been resolved, the process returns to step S101.

  When the personal computer CPU 210 proceeds to step S119, it determines whether or not the user has created a sentence such as an e-mail. If the personal computer CPU 210 determines that a sentence has been created, the process proceeds to step S120. If it is determined, the process returns to step S101.

  When a user is creating a sentence such as an e-mail when the user is frustrated, the user may regret because the expression is too tight or a dangerous phrase that hurts a person is used. Therefore, in this embodiment, the ROM 212 stores an inappropriate expression as well as an expression obtained by correcting the inappropriate expression, and the expression corrected by the personal computer CPU 210 when the inappropriate expression is used in an irritated state. Change to

  In the case of e-mail without urgency, adjustment is made so that the user can examine the appropriateness of the expression when the user's frustration is settled without immediately sending the e-mail. Thereby, it is possible to prevent the human relationship from deteriorating.

  If the personal computer CPU 210 detects an inappropriate expression, it displays that fact on the display 201. When the user is having a conversation while transmitting and receiving video using the videophone function of the personal computer 200, the personal computer CPU 210 interrupts the transmission of the video, or corrects the prompt by changing the frequency of the speech. You can also. Instead of interrupting the video, image processing such as reducing the number of transmission pixels may be performed. When the execution of various adjustments is completed in step S120, the personal computer CPU 210 returns to step S101 again.

  Next, a case where the degree of concentration exceeds the threshold value as determined in step S104 will be described.

  When proceeding to step S105, the personal computer CPU 210 causes the timer 211 to start measuring time. The personal computer CPU 210 obtains the time to continue in a state where the user's concentration is high by the time measurement started in step S105. By this time measurement, it is possible to extract data on how much the user can maintain high concentration. Further, in the present embodiment, for example, when the degree of concentration is high throughout a predetermined time such as 90 minutes, as described later, a high degree of concentration continues for the user for a long time. Warning.

  Subsequently, in step S106, the personal computer CPU 210 restricts contact with the user. In particular, a restriction signal for restricting a contact request is transmitted to a two-way communication device in which a third party requests a contact to the user. Here, the telephone 400 will be described as an example of the bidirectional communication device.

  The personal computer CPU 210 transmits a control signal for stopping the ringing of the telephone 400 and setting the absence mode to the telephone 400. The telephone 400 receives the control signal, sets the ringer volume to 0, and sets the absence mode. When the telephone 400 receives a contact from a third party in the absence mode, the telephone 400 sends a contact request by e-mail or a message indicating that the call will be returned later along with the user's situation. It is also possible to ask the other party about the degree of urgency, and in the case of an emergency, for example, the number 1 is pressed, and only in that case, the user can be set to follow.

  The bidirectional communication device is not limited to an external device. When the personal computer 200 has a videophone function, for example, the contact request is restricted for the videophone function. Further, the contact request may be limited for the mail function provided as software in the personal computer 200. For example, in a normal state, when it is set to notify the reception by opening a pop-up window together with the reception of the mail, the pop-up window is restricted from being opened in the user's concentrated state. Alternatively, it can be set not to receive mail in a concentrated state. When the contact request of the two-way communication device is restricted, the display 201 may notify the user so that the user can recognize it.

  Moreover, the control signal which restrict | limits the contact to a user can be transmitted not only to a bidirectional | two-way communication apparatus but to various control apparatuses. In the present embodiment, control for the indicator light 410 will be described.

  As described above, in the indicator lamp 410, the light emission color represents whether or not contact with the user is permitted. For example, it is well known that red light emission means contact prohibition to the user, and blue light emission means contact permission. In step S106, the personal computer CPU 210 transmits a control signal indicating that red light emission is to be performed to the indicator lamp 410. In response to the control signal, the indicator lamp 410 emits red light. Thereby, the surrounding person can recognize that the user is in a concentrated state and understands that the contact is prohibited.

  In addition to the indicator lamp 410, for example, a control signal can be transmitted to a partition capable of controlling the liquid crystal surrounding the user. The personal computer CPU 210 can control such a partition to be in a non-transparent blindfold state when the user is concentrated, and to be in a transparent state during normal times. Further, the personal computer CPU 210 can also send a control signal for starting noise removal to a noise removing device that removes noise by generating sound waves having an opposite phase to noise from the surrounding environment. Furthermore, the personal computer CPU 210 can also send a control signal for locking the key of the user's room to the key control device.

  The personal computer CPU 210 proceeds to step S107, confirms the user's schedule, and confirms whether there is a work other than desk work such as a meeting. Then, when there is a schedule, the personal computer CPU 210 determines whether or not this schedule can be changed (step S108). For example, in the case of a conference, the personal computer CPU 210 determines whether or not the participating members of the conference are superior, whether or not the user is an essential attendee, whether or not the conference is highly urgent, and the like. . The personal computer CPU 210 determines that the change can be made if the attendee is not an essential attendee, or is not urgent even if the attendee is an essential attendee, and the superior does not participate. On the other hand, the personal computer CPU 210 determines that the change is not possible when there is an urgent need, or when the senior attendee is an essential attendee. The determination criteria are set in advance and recorded in the ROM 212 as a lookup table.

  The personal computer CPU 210 proceeds to step S109 if the determination in step S108 is YES, and proceeds to step S110 if NO. In step S109, the personal computer CPU 210 delivers an e-mail to the conference organizer and participants to automatically notify that the conference cannot be attended. Further, the personal computer CPU 210 displays on the display 201 that the conference has been canceled and allows the user to recognize it.

  Subsequently, the personal computer CPU 210 determines whether 90 minutes have elapsed since the start of time measurement in step S105 (step S111). If it is less than 90 minutes, the personal computer CPU 210 returns to step S101, and if it has reached 90 minutes, proceeds to step S112, and displays a warning on the display 201 that the state of high concentration continues for a long time. I do.

  The warning display performed in step S112 needs to be more conspicuous than the display when the conference is canceled in step S109. Therefore, the personal computer CPU 210 displays larger than the display when the conference is canceled in step S109, displays it longer in time, or executes blinking display.

  Further, the personal computer CPU 210 proceeds to step S113, applies a current to the temperature control unit 208 to warm the user's palm, and makes the user recognize the warning sensibly. When it is detected that the mouse 300 is used, a current may be applied to the temperature control unit built in the mouse 300. Further, the personal computer CPU 210 may transmit a cooling instruction control signal to the air conditioner to lower the ambient temperature.

  On the other hand, if the determination in step S108 is NO, the personal computer CPU 210 proceeds to step S110, for example, displays on the display 201 that the conference is held five minutes before the start of the conference, and alerts the user. The display in this case is displayed larger than the display when the conference is canceled in step S109, displayed longer in time, or displayed in a blinking manner. Further, the personal computer CPU 210 proceeds to the above-described step S113, and uses the temperature control unit 208 or the like to make the user recognize the schedule opening sensibly.

  When the user exits from a highly concentrated state through the above-described series of processing, processing such as releasing the contact restriction is executed, and the flow ends. In the above flow, the time for executing the contact restriction is set as a time to be alerted or a time until a schedule is held while maintaining a high degree of concentration, but is not limited thereto. For example, you may set the time which a user wants to concentrate beforehand. If comprised in this way, the user can interrupt | block contact with the exterior by own intention. Or personal computer CPU210 can extract the average concentration time which can be continued for every user from a past log, and can also set this. By adopting such a configuration, it is possible to perform control according to the characteristics of each user.

  In the above-described embodiment, facial expressions are detected by imaging the user's face with the built-in camera 204. Instead of this or in combination, the movement of the user's hand can be used as a judgment material. For example, hand reference images corresponding to various movements are prepared in advance, and the image analysis unit 215 performs pattern matching processing with an image captured by the built-in camera 204 to recognize hand movements.

  FIG. 4 is a flowchart showing processing relating to detection of a user's hand as an application example of the first embodiment. Specifically, it is a flowchart showing detection of a user's hand by control of the personal computer CPU 210 and processing according to the detected state of the hand. In addition, the hand part said here is not only the hand itself but also a part including the wrist and the arm near the wrist.

  Step S201 is a process for inputting biometric information, and is substantially the same as the process in step S101. Step S202 is an analysis of the imaging result, which is substantially the same as the processing in step 102. Therefore, description of these processes is omitted.

  In step S <b> 203, the personal computer CPU 210 determines from the analysis result of the image analysis unit 215 whether the image signal captured by the built-in camera 204 includes the keyboard 202 and the user's hand. Specifically, the image analysis unit 215 first analyzes whether or not at least a part of the keyboard 202 overlaps at least a part of the user's hand. Further, the image analysis unit 215 may analyze whether the overlapping hand portion is the right hand or the left hand, whether the hand portion overlaps the mouse 300, or the like. As an analysis result, the image analysis unit 215 delivers position information including the relative positional relationship between the keyboard 202 and the user's hand to the personal computer CPU 210. The personal computer CPU 210 can predict the user's operation based on the position information delivered from the image analysis unit 215 even before the user actually operates the keyboard 202 or the like. If the personal computer CPU 210 determines from the delivered position information that the keyboard 202 and the user's hand are overlapping, the process proceeds to step S204. If the personal computer CPU 210 determines that there is no overlap, the process skips step S204 and proceeds to step S205.

  In step S204, the personal computer CPU 210 adjusts the operation unit settings. Specifically, the personal computer CPU 210 changes the reaction speed of the keyboard 202 and the touch pad 203. Note that the reaction speed here includes the concept of sensitivity to key touch, such as whether it reacts even with a slight touch or does not respond unless it is touched firmly. Here, since the keyboard 202 and the user's hand overlap each other and it is presumed that the keyboard 202 is operated as the user's intention, the personal computer CPU 210 determines the reaction speed (sensitivity) of the touch pad 203 as an adjacent operation unit. To slow down. In other words, it is set so that it does not react unless it is touched firmly. Alternatively, the personal computer CPU 210 may change the setting so that an input operation to the touch pad 203 is not accepted. As a result, it is possible to prevent malfunction when the user's hand and its vicinity accidentally approach the touch pad 203. As to which setting is to be changed, the personal computer CPU 210 can adopt conditions such as whether or not both hands of the right hand and the left hand overlap the keyboard 202.

  Furthermore, the biometric information acquired in step S201 may be taken into account as a setting change condition. For example, when the emotion analysis unit 213 detects the user's frustrating state, the personal computer CPU 210 changes the setting so that the reaction speed with respect to the input to the keyboard 202 becomes faster. Further, the reaction rate may be changed according to the degree of irritation. In this case, for example, settings of 2 to 4 steps are prepared in advance as speed steps. When the adjustment of the operation unit setting is completed, the personal computer CPU 210 proceeds to step S205.

  In step S205, the personal computer CPU 210 determines from the analysis result of the image analysis unit 215 whether the image signal captured by the built-in camera 204 includes the touch pad 203 and the user's hand. The personal computer CPU 210 can predict the user's operation based on the position information delivered from the image analysis unit 215 even before the user actually operates the touch pad 203 or the like. If the personal computer CPU 210 determines from the delivered position information that the touch pad 203 and the user's hand overlap, the personal computer CPU 210 proceeds to step S206. If it determines that the touch pad 203 does not overlap, the personal computer CPU 210 skips step S206 and proceeds to step S207.

  In step S206, the personal computer CPU 210 adjusts the operation unit setting. Specifically, the personal computer CPU 210 changes the reaction speed of the keyboard 202 and the touch pad 203. Specifically, the personal computer CPU 210 returns to the original setting when the response speed of the touch pad 203 is slowed down or not accepted by the adjustment in step S204. In particular, when it is determined that the keyboard 202 and the user's hand do not overlap, the personal computer CPU 210 may restore the original setting. Furthermore, the personal computer CPU 210 may increase the reaction speed of the touch pad 203 when the touch pad 203 is continuously operated. Moreover, you may consider the biometric information acquired by step S201 as conditions for a setting change. For example, when the response speed of the keyboard 202 is increased in consideration of biological information, only the setting of the touch pad 203 can be changed while the setting of the keyboard 202 is maintained. When the adjustment of the operation unit setting is completed, the personal computer CPU 210 proceeds to step S207.

  In step S207, the personal computer CPU 210 determines whether or not the input operation by the user has been completed. Specifically, when the input to the keyboard 202 and the touch pad 203 is not allowed for a predetermined time, it is determined that the input operation has been completed. The personal computer CPU 210 returns to step S201 when determining that the input operation is continued, and ends the series of processing flows when determining that the input operation has been completed.

  In the above flow, the user's hand, the keyboard 202, and the touch pad 203 are photographed by the built-in camera 204, but may be photographed by the ceiling camera 320. In the above flow, the description has been made on the assumption that the user operates the touch pad 203. However, when the mouse 300 is operated instead, the setting for the touch pad 203 is changed to the setting of the mouse 300. It can be replaced and applied. In this case, the personal computer CPU 210 may set the touch pad 203 so as to slow down the reaction speed or not accept input.

  In the above flow, the user's hand, the keyboard 202, and the touch pad 203 are photographed by the built-in camera 204 to acquire position information. However, the operation is performed after receiving an actual input operation without using an image signal. The part setting may be changed. Since the setting of the operation unit is changed after the actual input operation is detected, a slight time lag occurs, but the processing load on the image analysis unit 215 can be reduced. In particular, it is effective to the extent that the operation sensitivity of the touch pad 203 is changed.

  Next, an application example using the voice analysis unit 214 will be described. FIG. 5 is a flowchart showing processing relating to detection of the user's speech rate as an application example of the first embodiment. Here, it is assumed that the user uses a videophone as a function of the personal computer 200.

  Step S301 is a process for inputting biometric information, which is substantially the same as the process in step S101, and a description thereof will be omitted. In step S302, the personal computer CPU 210 analyzes the image signal from the built-in camera 204 by the image analysis unit 215 to detect the facial expression of the user. Furthermore, the personal computer CPU 210 determines the user's mood from the user's facial expression.

  In step S303, the personal computer CPU 210 analyzes the voice signal from the microphone 207 by the voice analysis unit 214 and detects the speech rate of the user. Specifically, the voice analysis unit 214 calculates the speech rate by counting the number of output phonemes per unit time.

  Subsequently, in step S304, the personal computer CPU 210 determines whether or not the speaking rate has increased beyond a predetermined threshold. That is, the personal computer CPU 210 captures the user's excitement using a physiological phenomenon in which the utterance speed increases rapidly in the initial stage of excitement. For example, the personal computer CPU 210 can continuously monitor the utterance speed at normal times and record it in the ROM 212, and can set the recorded utterance speed at normal time plus 20% as a threshold value. It should be noted that each user can be identified by face recognition technology or the like, and a threshold for each user can be set.

  In addition, the personal computer CPU 210 can take into account at least one of the information in step S301 and step S302 as the determination condition. For example, the personal computer CPU 210 does not determine that the utterance speed has increased even if the utterance speed has increased beyond the threshold, if it is determined that the mood is good by facial expression detection. Or personal computer CPU210 can add that negative emotions, such as "excitement", "irritability", and "impression", are detected by biometric information as conditions which judge that speech speed rose. The combination of these information can also be comprehensively determined by weighting each detection result.

  The personal computer CPU 210 returns to step S301 when determining that the increase amount of the speech rate is less than the threshold value, and proceeds to step S305 when determining that it exceeds the threshold value.

  When proceeding to step S305, the personal computer CPU 210 executes various adjustments. First, the personal computer CPU 210 allows the user to recognize the fact that it is quickly spoken and eventually the fact that it is in an excited state. Specifically, the personal computer CPU 210 reduces the display brightness of the display 201 to darken it. Alternatively, a message to that effect is directly displayed on the display 201 as text or a figure.

  In addition, a control signal can be transmitted to the external device so that the user can recognize the external device. Specifically, the personal computer CPU 210 transmits a control signal to the indicator lamp 410 to blink the LED. In addition, a control signal is transmitted to the lighting device installed in the user's room to change its brightness and change the brightness of the room. Furthermore, the output sound of a television, music player, etc. around the user can be reduced.

  In addition, in the user's excitement state, it can be expected that the personal computer CPU 210 positively restricts the communication state of the videophone, thereby preventing the deterioration of the human relationship of the user. Specifically, the personal computer CPU 210 can change or interrupt the video of the other party in the videophone. In addition, the user's voice to be transmitted can be processed. For example, the frequency can be changed so that it can be heard gently. Alternatively, the personal computer CPU 210 may deteriorate the communication state of the videophone and eventually cut off the communication.

  The personal computer CPU 210 proceeds to step S306 and starts recording at least one of the image signal from the built-in camera 204 and the audio signal from the microphone 207. An image signal from the ceiling camera 320 may be recorded. Thus, by recording the user's surrounding environment including the user in the user's excitement state, the user's memory can be supplemented and the communication record can be reliably left. Further, the user can objectively look back and recognize his / her behavior when returning to the normal state.

  In the first embodiment described above, the concentration level detection system 110 has been described by taking the operation of the personal computer 200 as an example, but instead, the concentration level detection system 110 may be applied to the operation of the smartphone.

  FIG. 6 is a diagram illustrating an overview of a smartphone 250 that is a modified example of the first embodiment. As shown in FIG. 6, the smartphone 250 has a vertically long rectangular shape, and includes a display 251, a touch panel 252 provided on the surface of the display 251, a built-in camera 254, a microphone 257, and a biosensor 260. Is provided.

  The touch panel 252 can accept various instructions when the user touches the surface of the display 251. The built-in camera 254 is provided on the same side as the touch panel 252 and includes a photographing lens and an image sensor. In addition to the built-in camera 254, a built-in camera may be provided on the side opposite to the touch panel 252.

  The microphone 257 is provided below so that the user can easily face the user's mouth when holding the smartphone 250. The biometric sensor 260 is provided on the side surface on the long side so as to touch the user's hand when the user holds the smartphone 250. The biosensor 260 may be provided in the main body of the smartphone 250, or a wristwatch-type biosensor 330 may be used as in the first embodiment described above.

  FIG. 7 is a block diagram of a concentration level detection system according to a modification of the first embodiment. The configuration other than the configuration described with reference to FIG. 6 can be substantially applied to the configuration of the block diagram of FIG. 2, and thus the same reference numerals as those in FIG. The smartphone CPU 270 is a control device that controls the entire smartphone 250.

  Also in this modification, the smartphone CPU 270 restricts contact with the user when the user is concentrated based on the user's biological information. In this case, in addition to the telephone call to the smartphone 250, when the user is in a company seat, the function of the telephone 400 in the seat may be restricted. In addition, the user may be using the ceiling camera 320 in the seat or moving. The built-in camera 254 is used as a wide-angle lens to detect a facial movement in addition to the user's facial expression, and the user's emotion and concentration level. May be detected. Similarly, the movement of the user's hand may be captured by the ceiling camera 320 or by the built-in camera 254 having a wide-angle lens.

  In addition, the smartphone CPU 270 changes the setting using software to increase the sensitivity of the touch panel 252 when the operation amount of the touch panel 252 is large, or when the force to operate the touch panel 252 detected by the piezoelectric sensor 209 is large. May be. The first embodiment described above and the modified example of the first embodiment can be combined or modified as appropriate.

  FIG. 8 is a diagram showing an overview of the concentration level detection system 120 according to the second embodiment. The concentration level detection system 120 in the present embodiment is configured by appropriately using each element used in the concentration level detection system 110 in the first embodiment. In addition, the concentration detection system 120 of the second embodiment has some elements added to the concentration detection system 110 of the first embodiment as described below. In particular, the personal computer 200 has substantially the same configuration in this embodiment, and a function of transmitting / receiving to / from an external device newly added to the first embodiment is added. Elements common to the first embodiment are denoted by the same reference numerals, and description thereof is omitted unless a new function is given.

  The concentration level detection system 120 according to the second embodiment is a system that detects the concentration level of participants in presentations, meetings, workshops, and the like and provides feedback to the participants. Unlike the first embodiment, a plurality of participants are targeted, and the degree of concentration of the subject is detected simultaneously or sequentially. In particular, here, a case of a lecture in which a presenter and a plurality of students are present as participants will be described as an example.

  Concentration detection system 120 is centered on personal computer 200, a ceiling camera 320, a presenter, a biosensor 330 attached to each of a plurality of students, a clock 500 installed on a wall, and a screen used by the presenter for presentations. A board 600 is provided. The ceiling camera 320 installed on the ceiling of the room has the same configuration as that of the ceiling camera 320 in the concentration detection system 110. Here, for example, the heads of a plurality of students participating in the lecture can be photographed. A wide-angle lens is used to adjust the shooting angle of view.

  In the image signal output from the image sensor of the ceiling camera 320, the coordinates of the pixel and the position of the lecture hall are associated in advance, and it is possible to grasp which seat of the lecture hall the photographed participant is located in. It is configured as follows. That is, the ceiling camera 320 serves as a position sensor that detects the position of the participant. If the lecture hall is large, a plurality of ceiling cameras 320 may be installed. In the lecture hall, assuming that the student is sitting on a chair, the height of the head is about 1200 mm to 1400 mm from the floor. Therefore, the ceiling camera 320 only needs to be focused with respect to this height.

  Moreover, the ceiling camera 320 can also photograph a student's hand. The personal computer 200 that obtains a photographed image from the ceiling camera 320 can grasp how a student puts his / her hand on a table during a lecture and takes notes or operates a laptop computer. In the case where the difference in the distance between the head and the hand from the ceiling does not fall within the depth of field of the ceiling camera 320, a configuration in which a focus lens is driven in the ceiling camera 320 may be employed.

  A clock 500 and a screen board 600 are installed on the wall of the lecture hall. The screen board 600 is installed in front of the participant table in the lecture hall, and is used for displaying presentation materials and the like. The watch 500 is installed not on the front of the participant table but on the side wall surface different from the installation surface of the screen board 600.

  The clock 500 includes a time display unit 510 that represents time and a clock camera 520 that captures at least students. The time display unit 510 is a clock that informs the participant of the current time, and may be an analog display or a digital display. The clock camera 520 is installed in the vicinity of the time display unit 510, and the shooting angle of view and the installation height are adjusted so that all the students participating in the lecture can be shot. Similarly to the ceiling camera 320, the clock camera 520 associates the pixel coordinates with the lecture hall position in the image signal output from the image sensor, so that the photographed participants can use the lecture hall. It is structured so that it can be grasped which seat it is located on.

  The screen board 600 includes a screen display unit 610 and a screen camera 620. The screen display unit 610 is a display unit that displays presentation materials and the like. The screen display unit 610 may be composed of a display element panel such as a liquid crystal or a combination of a projector and a projection screen. In addition, a display medium such as a white board may be used instead of an electrical display device. When using a non-electrical device such as a whiteboard, the presenter does not display the presentation material, but the presenter writes on the board with a marker or the like.

  The screen camera 620 is installed in the vicinity of the screen display unit 610, and the shooting angle of view and the installation height are adjusted so that all the students participating in the lecture can be shot. Similarly to the ceiling camera 320, the screen camera 620 associates pixel coordinates and lecture hall positions in advance in the image signal output from the image sensor. It is structured so that it can be grasped which seat it is located on.

  FIG. 9 is a block diagram of a concentration level detection system according to the second embodiment. For example, a recording unit 217 capable of recording a large amount of data composed of an HDD and an SSD is added to the personal computer 200. The recording unit 217 records an image signal sent from each camera, and records the analyzed participant data.

  Further, the personal computer CPU 210 acquires the biometric information from the biometric sensor 330 for the participant through the external connection interface 216 while identifying the biometric information with an ID or the like. Information from the floor sensor 310 is acquired in the same manner.

  The clock 500 includes a time display unit 510, a clock camera 520, a frequency detection unit 540, a recording unit 550, and an external connection interface 560 with the clock CPU 530 as a center.

  The clock CPU 530 controls the clock 500 as a whole. The frequency detection unit 540 detects the frequency at which the student views the clock 500. Specifically, an image signal captured by the clock camera 520 is received and analyzed to detect how many times the clock 500 has been viewed within a predetermined unit time for each student. In particular, since the watch 500 is installed on the side wall, the watch camera 520 cannot photograph the face of the student from the front when the student is looking at the screen display unit 610. Therefore, the frequency detection unit 540 detects that the student's face is directed to the time display unit 510 by using a face recognition technique. The frequency detection unit 540 determines that the time display unit 510 has been viewed when, for example, both eyes of the student are detected in order to accurately recognize whether or not the student's face is facing the time display unit 510. You may do it.

  Thereby, a student's concentration degree can be judged. That is, the personal computer CPU 210 can determine the degree of concentration of the student by receiving the frequency information detected by the frequency detection unit 540 from the clock CPU 530. The frequency information can take various variations. The frequency detection unit 540 can also construct the frequency information for each student by distinguishing each student, and counts as a frequency detection target when any of the students turns their face to the time display unit 510 By doing so, it is also possible to construct frequency information without distinguishing students. According to the former frequency information, it is possible to observe the distribution of students having a low degree of concentration by corresponding to the seat position, as will be described later. According to the latter frequency information, it is easy to observe the degree of concentration of the entire student.

  In addition, the frequency detection count can be changed for a specific student. For example, when there is a target person to be distinguished such as an important person among the students, the count value is weighted in correspondence with the seating position of the specific student. For example, one count is counted as 1.5. Alternatively, the counting of other students may be canceled and only specific students may be counted. By constructing such frequency information, the personal computer CPU 210 can grasp the interest level of an important person. Note that the seating position of the specific person is determined in advance, and even when seated in an arbitrary seat, the seating position of the specific person can be specified by face recognition using a captured image captured by the screen camera 620.

  Further, the frequency information can take into account the duration when the student's face is directed to the time display unit 510. The frequency detection unit 540 detects the duration directed to the time display unit 510, and when the time display unit 510 is viewed for a long time, the frequency detection unit 540 adds and counts the count values. Thereby, the concentration degree can be expressed more accurately.

  Note that the concentration of the students may be determined not by the personal computer CPU 210 but by the clock CPU 530. In this case, the clock CPU 530 transmits a control signal for controlling the external device to the external device via the external connection interface 560 according to the concentration level of the students. At this time, the student's biometric information may be received from the personal computer CPU 210 in advance and used as a condition for determining the degree of concentration. Specific control of the external device will be described later.

  The screen board 600 includes a screen display unit 610, a screen camera 620, and an external connection interface 640 with a screen CPU 630 as a center.

  The screen CPU 630 controls the entire screen board 600. As described above, the screen camera 620 can photograph all the students who participate in the lecture. In particular, since it is installed in the vicinity of the screen display unit 610, it is possible to detect whether or not the student's face is directed toward the screen display unit 610 using face recognition technology. Here, the screen CPU 630 transmits the image signal captured by the screen camera 620 to the frequency detection unit 540 of the watch 500 via the external connection interface 640.

  Similar to the analysis of the image signal from the clock camera 520, the frequency detection unit 540 detects, for each student, how many times the screen display unit 610 has been viewed within a predetermined unit time. Here, instead of the frequency of how many times it has been seen, the duration time may be measured and the gaze time per unit time may be detected. Thereby, contrary to the case where the clock camera 520 catches the student, it is possible to grasp in real time how concentrated each student is taking the lecture. That is, the personal computer CPU 210 can determine the degree of concentration of the student by receiving gaze information including the frequency or gaze time detected by the frequency detection unit 540 from the clock CPU 530.

  The counting process for the image signal received from the screen camera 620 in the frequency detection unit 540 is the same as the counting process for the image signal acquired from the clock camera 520. For example, when both eyes of the student are detected, it can be determined that the student has faced the screen display unit 610, and the count value can be weighted in correspondence with the seating position of the specific student. . When identifying and recognizing a person for each student, person recognition is performed by pattern matching between a reference image of a person image recorded in advance in the recording unit 217 and a captured image.

  In addition, the image signals photographed by the clock camera 520 and the screen camera 620 are transmitted to the image analysis unit 215, and the image analysis unit can analyze the image signals and detect the facial expressions of the students in the image. . The personal computer CPU 210 and the clock CPU 530 can be used as materials for various judgments with reference to the facial expressions of the students. Note that not only the personal computer 200 but also the clock 500 and the screen board 600 may each include an image analysis unit.

  In the present embodiment, it is assumed that the presenter and the participant of the student each wear the biosensor 330, but depending on the other party such as a customer, for example, a request for wearing the biosensor 330 may be hesitant. Therefore, a non-contact type biosensor can be used instead of the wearing type biosensor 330. For example, the body temperature change of the participant can be acquired by using thermography. In addition, biometric information from the voice of the student collected by the microphone 207 may be detected. In this case, instead of the personal computer 200, the microphone 207 may be appropriately provided with a highly directional microphone that can be identified for each participant in the lecture hall. Further, a floor sensor 310 embedded in the floor may be used.

  The concentration level detection system 120 is connected to various control devices installed in the lecture hall. For example, a lighting device that adjusts the brightness, a noise removal device that removes noise, and an air conditioning device that adjusts the temperature of the lecture hall. The personal computer CPU 210 can be controlled by transmitting a control signal to these devices. The control process will be described below.

  FIG. 10 is a flowchart showing processing of the concentration level detection system 120 according to the second embodiment. The flow is started, for example, from the time when the presenter starts the presentation.

  In step S401, the personal computer CPU 210 performs image input from the built-in camera 204 and ceiling camera 320 to the presenter, voice input from the microphone 207, biometric information input from the biosensor 330, and the like, and confirms the presenter's state. Specifically, the input information is analyzed by the emotion analysis unit 213, the voice analysis unit 214, and the image analysis unit 215, and the personal computer CPU 210 determines whether the presenter is in a tense state or a relaxed state.

  Further, the personal computer CPU 210 proceeds to step S402 and confirms the state of the student. In particular, the personal computer CPU 210 confirms the degree of concentration of the student from various pieces of input information. In response to the image signal from the ceiling camera 320, it is detected whether or not there is a student having a large head movement amount among the students, and if so, where the seat position is. Further, as described above, frequency information for viewing the time display unit 510 from the acquired image of the clock camera 520 and frequency information for viewing the screen display unit 610 from the acquired image of the screen camera 620 are acquired for each student.

  The presentation material video of the presenter displayed on the screen display unit 610 is based on the image signal transmitted from the personal computer CPU 210. Accordingly, the personal computer CPU 210 determines whether or not the student has turned the paper material at hand in response to the timing at which the presenter performs page feed of the presentation material by operating the personal computer 200. It can be judged from. If the student turns the paper material in a timely manner, it can be determined that the student is concentrating on the lecture. On the other hand, if the student's hand cannot be confirmed on the table, or the paper turning cannot be confirmed, it is highly possible that the students are not concentrated. The personal computer CPU 210 determines that the user is concentrated, for example, when the student performs the paper turning within 5 seconds from the page turning time by the presenter. Further, the personal computer CPU 210 not only synchronizes with the page feed of the presenter, but also determines, for example, that the student's hand is taking a note when there is movement on the table. You can also check regularly.

  The degree of concentration of the students is comprehensively determined by collecting various information as described above. For example, the personal computer CPU 210 applies various collected information to a lookup table stored in the ROM 212, acquires a concentration evaluation value for each information, and concentrates when these integrated values exceed a predetermined threshold value. It can be determined that Moreover, even if it is a case where it is less than a threshold value, the degree which is not concentrated can be grasped | ascertained by the number of the integrated value.

  After confirming the state of the student, the personal computer CPU 210 proceeds to step S403 and determines whether or not there is a student having a low concentration level. Note that the degree of concentration as the threshold value here can be set as the degree of concentration when not concentrated as described above. For example, at the time when the end of the lecture is reached, the degree of concentration as a threshold may be slightly reduced from the start of the lecture in consideration of the inevitably lower concentration.

  If there is a student with a low degree of concentration, the personal computer CPU 210 proceeds to step S406. In step S406, the personal computer CPU 210 confirms whether or not recording of at least one of the image signal from the built-in camera 204 and the sound signal from the microphone 207 has already been started. That is, it is confirmed whether the presenter's video and audio are being recorded. The reason for recording the lecture status in this way is to follow up so that students who are not concentrated can confirm the lecture again later. If recording is not in progress, the personal computer CPU 210 proceeds to step S407, starts recording, and proceeds to step S408. If recording is in progress, the process skips step S407 and proceeds to step S408.

  If there is no student with low concentration in step S403, the personal computer CPU 210 proceeds to step S404. In step S404, as in step S406, the personal computer CPU 210 checks whether recording is in progress. In this case, it is judged that the students are concentrated, and it is not necessary to record a lecture for follow-up. Accordingly, if recording is in progress, the personal computer CPU 210 proceeds to step S405, stops recording, and then proceeds to step 411. If not, the process proceeds to step S411.

  In step S <b> 408, the personal computer CPU 210 determines whether or not the student's concentration level is kept low. That is, it is determined whether or not there is a person with a low degree of concentration among the students even after a certain time has elapsed since the previous determination.

  If it is determined for the first time that the concentration level is low, or if it is determined that the concentration level is low again after the concentration level is recovered, the personal computer CPU 210 proceeds to step S409.

  In step S409, the personal computer CPU 210 detects the correlation between the less concentrated student and the seat position. As shown in FIG. 11, the correlation is graphically displayed on the management window displayed on the display 201, for example, as the concentration distribution, the seating position of the student and the less concentrated student. In the figure, white circles represent a group of students having a high degree of concentration, and black circles represent a group of students having a low degree of concentration. On the screen, the number of students having a low concentration with respect to the total number of students is shown as a number. In the state shown in FIG. 11, no tendency is seen in the seating positions of the students with low concentration, and the personal computer CPU 210 determines that there is no correlation between the students with low concentration.

  The personal computer CPU 210 proceeds to step S410 and executes various adjustments. For example, even if it is determined that there is no correlation with the seating position, if there are more than a threshold number of less concentrated students in the entire lecture hall, the personal computer CPU 210 controls the air conditioner to lower or raise the temperature. To perform temperature adjustment. For example, if the concentration of students on the corridor side is low with respect to the lecture hall, the personal computer CPU 210 sends a control signal for outputting an anti-phase sound wave of noise to the noise removing device that cancels noise from the corridor. Send to remove noise. For example, when the shaking of the head is large and the student is suspected of falling asleep, the personal computer CPU 210 transmits a control signal for brightening the lighting device to brighten the lecture hall.

  The personal computer CPU 210 proceeds to step S411 and determines whether or not the lecture has ended. If it is determined that the processing has not ended, the process returns to step S401. If it is determined that the processing has ended, the series of processing ends.

  If the personal computer CPU 210 determines in step S408 that the student's concentration level is continuously low, it proceeds to step S412 as YES. Here, the continuously low degree of concentration refers to, for example, a case where a plurality of predetermined specific students are determined to have a low degree of concentration continuously. Or even if a specific student does not continue, it may be a case where it is judged that one of the students with high importance continues and the concentration is low.

  In step S412, the personal computer CPU 210 determines whether there is an adjustable device that changes the environment of the lecture hall as in step S410. If there is not, the process proceeds to step S409, and if not, the process proceeds to step S413.

  When the process proceeds to step S413, a request for the presenter is made on the assumption that the decrease in the concentration of the student is not caused by the environment but is caused by the presenter's presenting operation. First, in step S413, as in step S409, the personal computer CPU 210 detects the correlation between the less concentrated student and the seat position. In this embodiment, the correlation detection in step S409 and step S413 is performed after the determination in step S408. However, the correlation detection may be performed before the determination in step S408.

  If correlation detection is performed, it will progress to step S414 and personal computer CPU210 will give an instruction | indication with respect to a presenter. For example, as shown in FIG. 12, when a correlation is obtained that a less concentrated student concentrates on the back seat of the lecture hall, the personal computer CPU 210 displays the management window displayed on the display 201 on the management window. , Display a message such as “Please make your voice louder”. Alternatively, the personal computer CPU 210 transmits a control signal for enlarging the size of the presentation material displayed on the screen display unit 610 to the screen board 600.

  At this time, the state of the presenter confirmed in step S401 can also be used. For example, when it is determined that the presenter feels “tension”, the personal computer CPU 210 displays the fact on the display 201 so as to objectively recognize the fact. Of course, information regarding the detected facial expression may be displayed. Further, in order to relieve tension, the order of the presentation materials may be changed, and the chat materials may be transmitted to the screen display unit 610. You can also change the animation process and display detailed materials.

  Furthermore, when the speech rate is detected and it is determined that the speech rate is faster than the threshold, the personal computer CPU 210 displays a message such as “Please speak a little more slowly” on the display 201 to the presenter. Can be made.

  When the instruction to the presenter is completed in step S414, the personal computer CPU 210 proceeds to step S411 and confirms the end of the lecture. When it is determined that the lecture has ended, the series of processing ends. Note that step S414 described above may be performed after the execution of various adjustments in step S410.

  In the above processing flow, students with a low degree of concentration at that time are detected, but the increase or decrease in the number of students with a low degree of concentration can also be used as a judgment criterion. In other words, in the detection of the degree of concentration performed twice with time, whether or not the number of students with low concentration has suddenly increased, and how the correlation with the seating position is used as a criterion. be able to.

  In the processing flow described above, the groups are separated according to the degree of concentration being high and low, but the groups can also be separated according to the emotional state of the student detected from the biological information. For example, the personal computer CPU 210 may execute various processes according to the distribution of the students who are in an “irritated state”. Various processes can also be executed using both the degree of concentration and the emotional state.

  The above processing flow has been described assuming a lecture, but the scope of application of the concentration level detection system 120 is not limited to this. For example, when applied to the workplace, the supervisor can recognize the degree of stress from the subordinate's biometric information and perform job assignment or voice call to prevent a decrease in workplace morale. Moreover, when it applies to a school lesson, the part which a student cannot understand can be grasped | ascertained and learning efficiency can be improved.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

110 Concentration Detection System, 120 Concentration Detection System, 200 PC, 201 Display, 202 Keyboard, 203 Touchpad, 204 Built-in Camera, 205 Ultrasonic Sensor, 206 Speaker, 207 Microphone, 208 Temperature Control Unit, 209 Piezoelectric Sensor, 210 PC CPU, 211 timer, 212 ROM, 213 emotion analysis unit, 214 audio analysis unit, 215 image analysis unit, 216 external connection interface, 217 recording unit, 250 smartphone, 251 display, 252 touch panel, 254 built-in camera, 257 microphone, 260 Biometric sensor, 270 Smartphone CPU, 300 Mouse, 310 Floor sensor, 320 Ceiling camera, 330 Biological sensor, 400 Telephone, 410 Indicator light, 500 Clock 510 time display unit, 520 clock camera, 530 clock CPU, 540 frequency detection unit, 550 recording unit, 560 external connection interface 600 screen board, 610 a screen display unit, 620 screen camera, 630 screen CPU, 640 external connection interface

Claims (11)

An utterance speed detector that detects the utterance speed of the target person;
An electronic device comprising: a control unit that controls a controlled device based on a detection result of the utterance speed detection unit.   The electronic device according to claim 1, wherein the control unit controls adjustment of luminance with respect to the controlled device that is a display device.   The electronic device according to claim 1, wherein the control unit controls the controlled device that is a display device to change a display image or stop display.   The electronic device according to claim 3, wherein the video displayed on the display device is a videophone video related to speech detected by the speech speed detection unit.   The electronic device according to any one of claims 1 to 4, wherein the control unit performs control to adjust luminance with respect to the controlled device that is a lighting device.   The electronic device according to claim 1, wherein the control unit controls sound output with respect to the controlled device that is a sound output device.   The electronic device according to claim 6, wherein the control unit performs control to change a frequency of the audio output. A facial expression detection unit for detecting the facial expression of the subject,
The electronic device according to claim 1, wherein the control unit performs the control in consideration of a detection result of the facial expression detection unit. A biometric information input unit for inputting biometric information that is information about the subject's living body,
The electronic device according to claim 1, wherein the control unit performs the control in consideration of the biological information. A recording unit for recording the surrounding environment of the subject by at least one of recording and recording;
10. The electronic device according to claim 1, further comprising a control unit that executes recording of the recording unit when the speech rate exceeds a predetermined rate. An utterance speed detection step for detecting the utterance speed of the target person;
An electronic device control program that causes a computer to execute a control step of controlling a controlled device based on a detection result of the utterance speed detection step.

Images