JP2012128523A - Information processing device and operating method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve operability in recognizing an instruction from a user.SOLUTION: An information processing device measures the frequency of blinking of a user, identifies the user of which frequency of the blinking satisfies a predetermined criterion as an operator, recognizes an instruction input which is different from blinking and is from the user identified as an operator, and executes processing according to the instruction input.

Description

  The present invention relates to a technique for recognizing an instruction from a user.

  2. Description of the Related Art Conventionally, an apparatus that recognizes a user's gesture (body movement) or a voice instruction and performs an operation according to the recognized instruction is known. In general, while using the device, the user moves his body or speaks in addition to giving instructions to the device. The following problems arise when there is a display device such as a display that operates in accordance with the recognized instruction. That is, when the user moves his hand while looking at the display device, eats or speaks with a person, the device makes the user's gesture or voice intentionally instruct the device. In some cases, it is impossible to determine whether the operation is unsuccessful, and an operation not intended by the user may be performed.

  As a solution to this problem, conventionally, there is a technique for causing a user to perform an operation of pressing a button or selecting a menu when performing gesture recognition or voice recognition. However, it is an advantage that gesture recognition can be used to operate the device using only hand movements, and voice recognition can be performed using only voices emitted from the mouth. That is, when other operations such as button pressing and menu selection are added, the operation becomes complicated for the user, and the advantages of gesture recognition and voice recognition may not be fully utilized.

  Therefore, as a method for solving this problem, a technique is known in which gaze on the user's device is determined as an instruction start signal, and gesture recognition and voice recognition are enabled according to the gaze target (Patent Document 1). . This technique detects a location, an area, a direction, and an object that the user is facing by using a camera or the like that observes the user, and makes recognition effective accordingly. In addition, this technique presents an anthropomorphic image in a part of the display area, and starts recognizing a voice uttered by the user when the user stares at the anthropomorphic image.

  On the other hand, conventionally, there is knowledge indicating the relationship between the object being visually observed and the number of blinks. For example, it is known that the characteristics of blinking change according to the psychological state of the user, the distance from the object being viewed, and the like. In particular, it is known that the number of blinks is extremely reduced when viewing a display terminal such as a television or a monitor (Non-Patent Document 1). In addition, when a stimulus that needs to be processed is presented, in the middle of processing, or when the stimulus is expected to be presented, blinking is suppressed, and when the processing is completed It is known that suppression is released and blinking occurs frequently. (Non-patent document 2).

Japanese Patent Laid-Open No. 10-301675

Tetsuko Namba, Sakiko Hotta, Akio Tanabe, “Relationship between the number of blinks and tear volume and the refractive correction method by Visual Display Terminal (VDT) work”, Kawasaki Medical Welfare Society, 2006 Takashi Matsuo, “Cognitive load at the time of hierarchical menu search -Analysis using blinks as an index-", 2000 Japan Directory Society research grant report, 2002

  However, in the method shown in Patent Document 1, it becomes difficult to determine the target that the user is staring at as the distance between the user and the device increases. The first reason that the determination becomes difficult is that the direction of the user's line of sight needs to be calculated with higher accuracy as the distance between the user and the device increases. Furthermore, as the distance increases, the difference between the direction of the user's line of sight when looking at one object and the angle of the user's line of sight when looking at another object decreases, and the user's eyes accordingly. This is because the difference in the image of the image becomes smaller. In addition, the second reason that it becomes difficult to distinguish is that as the distance between the user and the device increases, the image of the user's eyes reflected on the camera attached to the device becomes smaller, making it difficult to calculate the direction of the fine line of sight. Because it becomes. Note that if the user's gaze target is misidentified, recognition is not effective even though the user is staring at the personified image, or recognition is effective when the user is viewing the content displayed on the device. In some cases, the operability is reduced.

  SUMMARY An advantage of some aspects of the invention is to improve operability when recognizing an instruction from a user.

  In order to achieve the above object, an information processing apparatus according to the present invention is an information processing apparatus that executes processing according to a user's instruction input, and includes a measurement unit that measures a user's blink frequency, and the blink. A means for specifying a user satisfying a predetermined standard as an operator and an instruction input different from blinking, recognizing the instruction input by the user specified as the operator, and performing a process according to the instruction input Recognition means to execute.

  According to the present invention, it is possible to improve operability when recognizing an instruction from a user.

The functional block diagram which shows the function structure of the information processing apparatus which concerns on this invention. The flowchart which shows the flow of the process which recognizes the instruction | indication from a user. The flowchart which shows the flow of the process which determines the blink, and the flow of the process which determines the frequency of blink. The figure which shows the log | history of the blink in each flame | frame of a moving image, the log | history of eyes | visual_axis, and the log | history of distance. The functional block diagram which shows the function structure of the information processing apparatus which concerns on this invention. The flowchart which shows the flow of the process which recognizes the instruction | indication from a user.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. Note that blinking means opening after closing the eyelid.

(First embodiment)
FIG. 1A is a functional block diagram showing a functional configuration of the information processing apparatus 101 according to the present invention. Note that a device for displaying a moving image such as the output unit 102 is connected to the information processing apparatus 101. The output unit 102 may include a speaker and output sound. The input unit 103 is configured by a camera or the like. The input unit 103 captures a moving image and stores each frame of the captured moving image in the operation data 104. The blink measurement unit 105 includes a CPU, a ROM, and a RAM (hereinafter referred to as a CPU or the like), and the CPU realizes the function by developing and executing a program stored in the ROM on the RAM. Also, the blink measurement unit 105 estimates a user shown in one frame of a moving image stored in the motion data 104, measures whether the user is blinking, and outputs the result to the blink history 106. The processing in the blink measuring unit 105 is as shown in FIG. 3A, and details will be described later. The recognition target specifying unit 107 is configured by a CPU or the like. Further, the recognition target specifying unit 107 specifies a user as a recognition target from the blink history 106, the line-of-sight history 110, and the distance history 112, and instructs the recognition unit 108 of the specified user. The recognition unit 108 is configured by a CPU or the like. The recognizing unit 108 recognizes an instruction to the apparatus by a user's gesture from one frame of the moving image stored in the operation data 104, and operates the output unit 102 according to the instruction. In addition, the recognition unit 108 enables gesture recognition of a specific user instructed from the recognition target specifying unit 107. The line-of-sight measurement unit 109 is configured by a CPU or the like. Further, the line-of-sight measurement unit 109 estimates a user appearing in one frame of the moving image stored in the motion data 104, measures the direction of the user's line of sight, and outputs the result to the line-of-sight history 110. The distance measuring unit 111 is configured by a CPU or the like. In addition, the distance measuring unit 111 estimates a user shown in one frame of a moving image stored in the operation data 104, measures the distance between the user and the device, and outputs the result to the distance history 112.

  FIG. 2 is a flowchart showing a flow of processing for recognizing an instruction from the user. The processing shown in this flowchart is executed every time the input unit 103 captures one frame of a moving image. First, in step S <b> 201, the input unit 103 stores one frame of a captured moving image in the operation data 104. Next, in step S <b> 202, the line-of-sight measurement unit 109 estimates the name of the user shown in the image representing the latest one frame among the moving images stored in the motion data 104. Further, it is determined whether the user's line of sight is directed to the input unit 103, and the result, the user's name, and the current time are stored in the line-of-sight history 110. In step S <b> 203, the distance measurement unit 111 estimates the name of the user shown in the image representing the latest one frame among the moving images stored in the motion data 104. In addition, this process is good also as a process which specifies each user so that it can identify with another user instead of the process which estimates a user's name. In step S 203, the distance between the user and the apparatus is measured, and the result, the user name and the current time stored in the operation data 104 are stored in the distance history 112. In step S <b> 204, the blink measuring unit 105 estimates the name of the user shown in the image representing the latest one frame among the moving images stored in the motion data 104. Further, it is determined whether the user is blinking, and the result, the user's name, and the current time are stored in the blink history 106. In step S <b> 205, the recognition target specifying unit 107 determines the blink frequency from the blink history 106.

  In step S206, the recognition target specifying unit 107 determines from the distance history 112 whether the average distance between the user and the device within a predetermined period is smaller than a predetermined threshold. If the average distance is smaller than the threshold, it is determined that the user's gaze can be accurately determined only from the direction of the line of sight, and the process proceeds to step S210. If it is larger than the threshold value, it is determined that the gaze cannot be accurately determined unless the gaze direction and the blink frequency are used, and the process proceeds to step S207. In step S207, the recognition target specifying unit 107 determines whether the blink frequency determined in step S205 is greater than a predetermined threshold. Further, it is determined from the line-of-sight history 110 whether the period during which the line of sight is directed to the input unit 103 within a predetermined period is greater than a predetermined threshold. If the blink frequency is greater than the threshold and the period when the line of sight is directed to the input unit 103 is greater than the threshold, it is determined that the user is staring at the input unit 103, and the process proceeds to step S208. When the blinking frequency is smaller than the threshold value or the period when the line of sight is directed to the input unit 103 is smaller than the threshold value, it is determined that the user is not staring at the input unit 103, and the series of processing ends. That is, in step S207, the recognition target specifying unit 107 determines whether or not the blinking frequency satisfies a predetermined criterion. In step S208, the recognition target specifying unit 107 notifies the recognition unit 108 of the name of the user who has determined that the input unit 103 is stared in step S207. That is, in step S208, the recognition target specifying unit 107 executes processing for specifying the operator. In step S209, the recognizing unit 108 enables the user's gesture recognition notified from the recognition target specifying unit 107, and ends the series of processes. In step S210, the recognition target specifying unit 107 determines, based on the line-of-sight history 110, whether the period during which the line of sight is directed to the input unit 103 within a predetermined period is greater than the threshold. If the period when the line of sight is directed to the input unit is greater than the threshold, it is determined that the user is staring at the input unit 103, and the process proceeds to step S208. If it is smaller than the threshold value, it is determined that the user is not staring at the input unit 103, and the series of processing ends.

  FIG. 3A is a flowchart for explaining the flow of the process executed in step S204 to determine whether the user is blinking. In step S204, this flowchart is executed for the number of users shown in the image representing the latest one frame among the moving images stored in the operation data 104.

  First, in step S301, the blink measuring unit 105 determines whether the user's eyes shown in the image representing the latest one frame are open. If it is open, the process proceeds to step S302. If it is closed, the process proceeds to step S303. In step S302, the blink measuring unit 105 determines whether the user's eyes shown in the image representing the previous frame are closed. If it is closed, the process proceeds to step S304. If it is open, the process proceeds to step S303. In step S303, the blink measuring unit 105 sets, in the blink history 106, that the user has not blinked at the time when the latest one frame was shot, and ends the series of processes. On the other hand, in step S304, the blink measuring unit 105 sets, in the blink history 106, that the user has blinked at the time when the latest one frame was shot, and ends the series of processes.

  FIG. 3B is a flowchart illustrating the flow of processing that is executed in step S205 and that determines the frequency of user blinking. In step S205, this flowchart is executed for the number of users shown in the image representing the latest frame among the moving images stored in the operation data 104.

First, in step S311, the recognition target specifying unit 107 sets a measurement period of the blink frequency. In step S <b> 312, the recognition target specifying unit 107 refers to the blink history 106 and acquires the number of blinks performed by the user up to a time traced back in the past by the measurement period set from the current time. In step S313, the recognition target specifying unit 107 calculates the blink frequency using the following mathematical formula, and ends a series of processes.
(Blink frequency) = (Number of blinks acquired in step S312) / (Measurement period determined in step S311)

  FIG. 4A is a diagram illustrating a relationship between each frame of a moving image obtained by photographing the user stored in the operation data and data stored in the blink history. For example, at 20: 59: 40.2 seconds on December 17, 2009, the eyes of the user Taro are closed in the latest frame, and 20:59:40 on December 17, 2009, which is the previous frame. The user Taro's eyes are open in a 1 second frame. Therefore, it is stored in the blink history that the user Taro has blinked.

  FIG. 4B is a diagram showing the blink history 106, the line-of-sight history 110, and the distance history 112. The distance threshold used in step S206 in FIG. 2 is 200 cm, the blink frequency threshold used in step S207 is 0.4 times per second, the eye direction threshold is 4 seconds in the past 5 seconds, and the blink measurement period used in step S311 is Suppose that it is 5 seconds. At this time, at 20: 59: 45.0 seconds on December 17, 2009, the distance history 112 indicates that the average distance between the device and the user Taro in the last 5 seconds indicated by 401 is 200 cm or more. Therefore, it determines with it not being smaller than a threshold value by step S206, and progresses to step S207. In step S207, the blinking frequency of the user Taro is 0.4 times / second, which is equal to or greater than the threshold value, and the line-of-sight direction is 4.2 seconds, which is equal to or greater than the threshold value, in 5 seconds. The process proceeds to step S208. In step S209, the recognition unit determines that the user Taro is a gesture recognition target (operator) and validates the recognition. When the recognition is valid, the information processing apparatus 101 recognizes the instruction input by the user specified as the operator, which is an instruction input different from the blinking, and executes processing according to the instruction input.

  As described above, according to the present embodiment, it is possible to determine that the user is staring at the input unit even when the distance between the user and the apparatus is long, using the frequency of blinking. Furthermore, according to this embodiment, it is determined that the user is staring at the input unit regardless of the distance between the user and the apparatus, using both the number of blinks and the direction of the line of sight. This is because when the user gazes at an object other than the display unit other than the input unit 103 and the output unit 102, the frequency of blinking does not change from when the user gazes at the input unit 103, so that the gesture recognition becomes effective. Resolve. Furthermore, according to the present embodiment, when the distance between the user and the apparatus is short, the blink frequency is not measured, only the direction of the line of sight is measured to determine gaze, and when the distance is far, the direction of the line of sight is blinked. The gaze is determined by measuring the frequency. Since it is necessary to measure the number of blinks within a predetermined period in order to measure the blink frequency, the measurement takes a certain amount of time. When gaze is determined only by the direction of the line of sight, the responsiveness of the device is improved.

  In the present embodiment, when a plurality of operators are detected, the information processing apparatus 101 may execute processing according to an instruction input such as a gesture by the plurality of persons. Further, when a plurality of operators are detected, the information processing apparatus 101 may cause the output unit 102 to output an image or sound for notifying the user that there are a plurality of operators. When a plurality of operators are detected, the information processing apparatus 101 may end the series of processes illustrated in FIG.

(Second Embodiment)
The first embodiment does not describe a method of disabling gesture recognition after enabling a certain user. The timing for disabling gesture recognition is as important as the timing for enabling gesture recognition. If the user invalidates the recognition even though the user intends to give an instruction to the apparatus, the user has to instruct the apparatus to make the recognition valid again, and the operational feeling is lowered. On the other hand, if the recognition is left valid even though the user does not intend to give an instruction to the device, a gesture made by intention other than the user's instruction to the device will be erroneously recognized, and the operational feeling will be lost. descend. Therefore, in the present embodiment, the timing for invalidating the gesture recognition is determined according to the change in the blinking frequency. In particular, the recognition ends at the timing when the blinking frequency increases. If the operation of the device using the gesture by the user is regarded as one process, the blinking of the user is suppressed while operating the device, and when the suppression is released at the stage where the operation of the device is completed, Conceivable. Therefore, the timing at which the blinking frequency is increased can be regarded as the timing at which the user finishes the operation of the apparatus. Furthermore, in the present embodiment, the frequency of blinking that occurs immediately after the recognition is validated is used as the threshold value of the blinking frequency when the recognition is invalidated. In general, the frequency of blinking varies depending on the surrounding environment such as the distance between the device and the user and the brightness around the user, and physical characteristics such as the user's age and physical condition, and thus it is difficult to always use a specific value. On the other hand, immediately after the recognition is made effective, it is considered that the flapping frequently occurs when the suppression is released. Therefore, the frequency of blinking at this time can be used to determine the timing when the user's operation is completed.

  FIG. 5 is a functional block diagram showing a functional configuration of the information processing apparatus 500 according to the present embodiment. The same elements as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted. The recognition end determination unit 501 is configured by a CPU or the like. The recognition end determination unit 501 determines whether the recognition should be invalidated based on the blink history 106 and the recognition end threshold 502 when the recognition unit 108 enables the recognition. When the recognition end unit 501 determines that the recognition should be invalidated, the recognition unit 108. Notify that the recognition is invalid. Specifically, if the user's blink frequency is greater than the recognition end threshold 502, the recognition unit 108 is instructed to invalidate the user's recognition. The recognition target specifying unit 107 measures the blink frequency used as a threshold from the blink history 106 after a predetermined period after the output unit 102 outputs that the user's recognition is valid, and sets it as the recognition end threshold 502. The reason for waiting for the elapse of the predetermined period is that a certain period is required from when the output unit 102 outputs that the user's recognition is validated until the user reacts and frequently blinks.

FIG. 6 is a flowchart showing the flow of processing for recognizing an instruction from the user in this embodiment. In addition, about the process similar to FIG. 2, the same code | symbol is attached | subjected and the description is abbreviate | omitted. In step S601, the recognition end determination unit 501 determines whether the recognition unit 108 has enabled recognition. If it is valid, the process proceeds to step S602. If not valid, the process proceeds to step S206. In step S602, the recognition target specifying unit 107 determines whether a predetermined period has elapsed since the recognition unit 108 enabled recognition. If it has elapsed, the process proceeds to step S603. If the predetermined period has not elapsed or the period exceeding the predetermined period has elapsed, the process proceeds to step S604. In step S603, the recognition target specifying unit 107 calculates the recognition end threshold value 502 using the following formula, and ends the series of processes.
(Recognition end threshold 502) = (number of blinks performed by the user within the immediately preceding predetermined period) / (predetermined immediately preceding period)

  In step S604, the recognition end determination unit 501 determines whether a predetermined period has elapsed since the recognition was validated. If it has elapsed, the process proceeds to step S605. If it has not elapsed, the series of processing ends. The reason for determining whether a predetermined period or more has elapsed in this step is that the user's blink that occurs frequently immediately after the recognition is validated is not erroneously determined as the recognition end timing. The reason why the user blinks frequently is that the user's process of staring at the input unit 103 and starting recognition by the apparatus is terminated, and the suppression of the user's blinking is released. In step S605, the recognition end determination unit 501 acquires the user name being recognized from the recognition unit 108 (identifies as a user). In step S 606, the recognition end determination unit 501 determines whether the blink frequency within the predetermined period immediately before is greater than the recognition end threshold 502 based on the blink history 106 associated with the user name acquired in step S 605. When it is larger than the threshold, it is determined that the user has finished the gesture operation, and the process proceeds to step S607. If it is smaller than the threshold value, it is determined that a gesture operation is being performed, and a series of processing ends. In step S607, the recognition end determination unit 501 notifies the recognition unit 108 of the name of the user who has determined that the gesture operation has ended in step S606. In step S608, the recognition unit 108 invalidates the user's gesture recognition notified from the recognition end determination unit 501, and ends the series of processes.

  Using the blink history 106, the line-of-sight history 110, and the distance history 112 shown in FIG. 4B, the flow of processing in which the recognition target specifying unit 107 sets the recognition end threshold value 502 in the present embodiment will be described. Note that the fixed period used in step S602 and step S603 is 5 seconds. It is assumed that the recognition of the user Taro is valid at 20: 59: 45.0 seconds on December 17, 2009. At 20: 59: 50.0 seconds on December 17, 2009, since the recognition is valid in step S601, the process proceeds to step S602. In step S602, since exactly 5 seconds have elapsed after the recognition is valid, the process proceeds to step S603. In step S603, since the number of blinks in the last 5 seconds indicated by 402 is 3, the threshold value 0.6 times / second is set as the recognition end threshold value 502.

  Next, using the blink history 106, the line-of-sight history 110, and the distance history 112 shown in FIG. 4B, in the present embodiment, when the recognition end determination unit 501 uses the threshold value of the recognition end threshold value 502 to end user recognition. The flow of processing for determination will be described. Note that the fixed period used in step S604 and step S605 is 5 seconds. Note that the recognition of the user Taro is effective at 20: 59: 45.0 seconds on December 17, 2009, and the user is recognized at 20: 59: 50.0 seconds on December 17, 2009. Assume that the Taro recognition end threshold 502 is set to 0.6 times / second. Since the recognition is valid in step S601 at the time of December 17, 2009, 21: 00: 25.0, the process proceeds to step S602. In step S602, since 5 seconds or more have elapsed after the recognition is valid, the process proceeds to step S604. In step S604, since 5 seconds after the recognition is valid is planned, the process proceeds to step S605. In step S605, the user name Taro being recognized is acquired. In step S606, since the blinking frequency for 5 seconds immediately before indicated by 403 is equal to or greater than 0.6 times / second stored in the recognition end threshold value 502, the process proceeds to step S607. If the user name Taro is notified to the recognition unit 108 in step S607, the recognition unit 108 invalidates gesture recognition in step S1605.

  As described above, according to the present embodiment, by using the user's blink frequency, the user's recognition can be invalidated at the timing when the user finishes the operation of the apparatus, and the operability is not deteriorated. . Furthermore, when the recognition is invalidated, the timing at which the user finishes the operation of the device can be determined more accurately by using an appropriate blink frequency threshold that takes into account factors such as the user's surrounding environment and body characteristics. Does not decrease the sex. Note that the recognition target specifying unit 107 can also change the threshold value of the blink frequency used for determining the gaze on the input unit 103 in step S207 according to the distance between the user and the display device. For example, referring to the distance history 112 in step S207 of FIG. 6, when the distance between the user and the display device is smaller than 100 cm, the user's blinking frequency is higher than normal because the distance between the user and the display device is short. Assumes few. Therefore, if the blinking frequency occurs once or more in the immediately preceding 5 seconds, the process proceeds to step S208. On the other hand, if the distance between the user and the display device is greater than 100 cm, the process proceeds to step S208 if the blinking frequency occurs twice or more in the immediately preceding 5 seconds. The recognition target specifying unit 107 can also change the target period for calculating the blink frequency according to the distance between the user and the device. Specifically, when the distance between the user and the apparatus is close and the accuracy of the measured gaze direction is high, the response to the user is accelerated by taking a short period. On the other hand, if the distance is long and the accuracy of the direction of the measured line of sight is low, a longer period is used to accurately measure the blink frequency. In addition, the recognition target specifying unit 107 instructs the input unit 103 to display the recognition target user in the center of the moving image to be captured in order to accurately recognize the user's gesture when the recognition unit 108 enables the user's recognition. It is also possible to do. For example, when the recognition target specifying unit 107 gives an instruction to the input unit 103 based on the position of the user measured by the position measurement unit that measures the position of the user from the operation data 104, the input unit 103 displays the front of the user. Operates to face. In addition to the recognition end threshold 502, the recognition end determination unit 501 invalidates user recognition based on the distance between the display device and the user and the brightness of the user's surroundings measured by the illuminance measurement unit that measures illuminance. It is also possible to determine whether or not. It is known that the frequency of blinking decreases as the distance between the display device and the user decreases. It is also known that the blinking frequency temporarily increases when the ambient brightness changes. By determining distance and brightness, it is possible to determine an appropriate blink frequency threshold that takes into account the user's surrounding environment, thereby more accurately determining when the user has finished operating the device. Does not decrease operability. Specifically, using the conditional expression shown below, in step S606 in FIG. 6, the recognition end determination unit 501 determines whether to invalidate the user's recognition.

  Here, a is the frequency of blinking (number of times / second), b is the recognition end threshold value 502, c is the current distance (cm), and d is the distance (cm) when the recognition is valid. is there. Also, e is the current illuminance (lux), f is the illuminance (lux) when recognition is valid, and g is the elapsed time (seconds) since the last change of illuminance. That is, c / d is a parameter related to the distance to the user. As the distance between the user and the device decreases, the frequency of blinking also decreases, so the threshold value is decreased accordingly. (1+ (e−f) / (f × g)) is a parameter relating to illuminance. When the illuminance around the user changes, the frequency of blinking increases as the amount of change (ef) increases, and the threshold value is increased accordingly. In addition, the increase in the blinking frequency due to the change in illuminance is temporary and returns to the original frequency as time elapses. Therefore, as the elapsed time g after the last change in illuminance increases, the illuminance parameter increases. To approach 1. For example, it is assumed that the recognition end threshold 502 is 1 time / second, the distance between the current display device and the user is 240 cm, and the distance between the display device and the user when recognition is valid is 300 cm. It is also assumed that the current illuminance is 600 lux, the illuminance when recognition is enabled is 150 lux, and the elapsed time since the last change of illuminance is 6 seconds or more. At this time, from the above conditional expression, the recognition end determination unit 501 determines that the user operation has ended when the user's blink frequency is 1.25 times / second in step S606 shown in FIG. In step S607, the recognition unit 108 invalidates the user's recognition.

  In addition to the recognition end threshold 502, the recognition end determination unit 501 can also determine whether to invalidate the user's recognition based on the user's line of sight or the voice emitted by the user measured by the voice measurement unit. is there. In particular, when the user's line of sight is not facing the device or the user is speaking, it is determined that the user is concentrating on other matters such as conversation, and the user's recognition is invalidated even if frequent blinking occurs. It is also possible not to. Specifically, in step S606 of FIG. 6, it is determined that the blinking frequency is equal to or greater than the threshold, the user's line of sight is facing the apparatus, and the user is not speaking. Only when these results are all positive, the process proceeds to step S607, and the recognition is invalidated in step S608.

  The recognition target specifying unit 107 can also instruct the output unit 102 to notify the user that the recognition has started after starting the user's recognition in order to inform the user that the recognition has started. is there. In particular, the output unit 102 may display an eye image to the user when the user's recognition is started, and output an animation in which the direction of the eye line of the eye image gradually faces the user. When a person notices that a person is staring at another person, the reaction of turning his gaze toward that person is very natural, so the display device outputs an image of the eye to the gaze, and the user who recognizes the gaze It is intuitive for the user to perform a reaction toward the user and enhances operability.

  Further, the recognition target specifying unit 107 can also determine a user who can be instructed using the priority order from the users whose gesture recognition is enabled at the same time. The priority order may be determined in advance by the user, or may be determined by comparing each user's blink frequency, line-of-sight direction, distance to the device, and position. In addition, it may be notified to the user via the output unit 102 which user can instruct the apparatus.

  Further, the input unit 103 can also delete a frame image of the moving image stored in the operation data 104. The blink measuring unit 105 can also delete the blink history stored in the blink history 106. Further, the output unit 102 may transmit various information to the user by generating a sound or operating another device. Further, the distance measuring unit 111 may record a voice uttered by the user, measure the temperature of the user, or measure the distance between the user and the apparatus using infrared rays or the like. The operation data 104 may store audio data, temperature data, and distance data. In addition to the gesture, the recognition unit 108 may recognize a voice or a user's posture as an instruction input to the apparatus.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (6)

An information processing apparatus that executes processing according to a user instruction input,
A measuring means for measuring a user's blink frequency;
Identifying means for identifying a user whose blinking frequency satisfies a predetermined criterion as an operator;
An information processing apparatus comprising: an instruction input that is different from blinking, and that recognizes an instruction input by a user specified as an operator and executes a process according to the instruction input.   The information processing apparatus according to claim 1, wherein the instruction input is an instruction input by a gesture.   The information processing apparatus according to claim 1, wherein the instruction input is a voice instruction input. Furthermore, it has a detecting means for detecting the direction of the user's line of sight,
The information processing apparatus according to any one of claims 1 to 3, wherein the specifying unit specifies, as an operator, a user whose blinking frequency and the direction of the line of sight satisfy a predetermined criterion. apparatus. An operation method of an information processing apparatus that executes processing according to a user's instruction input,
A measurement process for measuring the frequency of user blinking;
A specific step of identifying a user whose blinking frequency satisfies a predetermined criterion as an operator;
An operation method including a recognition step of recognizing an instruction input by a user specified as an operator, and executing a process according to the instruction input, which is an instruction input different from blinking.   A program for causing a computer to execute the operation method according to claim 5.

Images