JP2014021893A - Information processing device, operation signal generation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device for realizing an input operation excellent in operability.SOLUTION: A line-of-sight information acquisition unit 31 acquires line-of-sight information indicating a line of sight of a user. A sound information acquisition unit 32 acquires sound information indicating a sound, and when volume of the sound is equal to or larger than a sound volume threshold, determines that an input position is selected. An operation signal generation unit 33 generates an operation signal indicating an input operation to an operation surface on the basis of the line-of-sight information and the sound information.

Description

  The present invention relates to an information processing apparatus, an operation signal generation method, and a program.

  An input operation to the information processing apparatus is mainly performed using a hand. For example, when the input device is a mouse, the input operation includes moving the mouse with a hand and pressing a button provided on the mouse with a hand. When the user moves the mouse, the input position is changed, and when the user presses the button, the input position at that time is selected. When the input device is a touch-type input device, the input operation is to bring an operation body such as a finger or a stylus held by the hand into contact with or close to the operation surface (hereinafter referred to as touch). The touch position corresponds to the selected input position in the case of the mouse, and the type of input operation is determined based on the touch position, the moving direction of the touch position, and the touching time.

  Such an input operation using a hand is widespread because it is intuitive and excellent in operability, but in recent years, there is a demand for an information processing apparatus capable of realizing an input operation using something other than a hand. Is growing. For example, with the widespread use of information processing devices such as smart phones and tablet terminals, the opportunity to use information processing devices while moving is increasing, but there are many situations in which users cannot move their hands freely while moving. It is desirable to reduce input operations using hands as much as possible. In particular, tablet terminals are becoming larger in size, and it is common to hold the terminal with one hand and perform input operations with the other hand. For this reason, when holding a luggage with one hand, an input operation using a hand is difficult, and an input operation using a hand other than the hand is required.

  Therefore, as a method for realizing an input operation using something other than a hand, a method of specifying an input position on the operation surface with a line of sight and selecting an input position by opening and closing the heel can be considered. In this method, the user specifies the input position by directing his / her line of sight on the operation surface, and selects the input position by closing the bag. In this case, for example, the user performs an operation similar to the click operation by blinking, or performs an operation similar to the long press operation by opening the bag after a predetermined time has elapsed with the bag closed once. can do.

  However, in the above method, even if the user tries to perform the same operation as the drag operation and moves the line of sight after closing the eyelid, the line of sight is not detected when the eyelid is closed, and the input position is selected. It is not possible to perform the move operation.

  Therefore, instead of opening and closing the bag, a method of determining that the input position is selected when looking at the same place for a certain period of time can be considered. However, with this method, the input position cannot be selected without waiting for a certain period of time, so that the operability is degraded.

  On the other hand, Patent Document 1 discloses a portable terminal that specifies an input position on an operation surface by line of sight and selects an input position by pressing a button. According to this portable terminal, when the user presses the button with his / her hand in a state where the line of sight is directed to the character on the display screen, the character displayed at the position where the line of sight is directed can be input.

  On the other hand, Patent Document 2 discloses a portable terminal that inputs information by an air flow such as a user's breath. According to this portable terminal, the type of input operation can be changed based on a pattern that combines the strength and length with which the user blows.

JP 2010-267071 A JP 2004-177992 A

  However, in the portable terminal described in Patent Document 1, the user must press the button with his hand to select the input position. For this reason, in the situation where both hands do not move freely, there is a problem that the operability is low, such as it becomes difficult to perform an input operation.

  Moreover, although the portable terminal of patent document 2 changes the kind of input operation with the pattern which combined the strength and length which a user blows, since this pattern is limited, complicated input operation is carried out. There is a problem that it cannot be performed and operability is low.

  An object of the present invention is to provide an information processing apparatus, an operation signal generation method, and a program that realize an input operation with better operability.

  An information processing apparatus according to the present invention operates based on a line-of-sight information acquisition unit that acquires line-of-sight information indicating a user's line of sight, a sound information acquisition unit that acquires sound information indicating sound, and the line-of-sight information and the sound information An operation signal generation unit that generates an operation signal indicating an input operation on the surface.

  An operation signal generation method according to the present invention acquires line-of-sight information indicating a user's line of sight, acquires sound information indicating sound, and generates an operation signal indicating an input operation on an operation surface based on the line-of-sight information and the sound information. Generate.

  The program according to the present invention includes a procedure for acquiring gaze information indicating a user's gaze, a procedure for acquiring audio information indicating a voice, and an operation signal indicating an input operation on the operation surface based on the gaze information and the audio information. And causing the computer to execute.

  According to the present invention, it is possible to realize an input operation with better operability.

It is a block diagram which shows the function structure of the information processing apparatus concerning the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the gaze information acquisition part concerning this embodiment. It is explanatory drawing which shows an example of the gaze detection image used in order to detect a user's gaze. It is explanatory drawing for demonstrating the edge detection process result of a gaze detection image. It is a block diagram which shows the structural example of the audio | voice information acquisition part concerning this embodiment. It is a flowchart for demonstrating an example of the determination process which determines the process performed by the control part concerning this embodiment by input operation which an operation signal shows. It is a flowchart for demonstrating an example of the determination process of input operation when an operation signal production | generation part produces | generates an operation signal based on eyes | visual_axis information and audio | voice information. It is a block diagram which shows the function structure of the information processing apparatus concerning the 2nd Embodiment of this invention. It is a flowchart for demonstrating the operation example of the operation mode setting part concerning this embodiment. It is a block diagram which shows the function structure of the information processing apparatus concerning the 3rd Embodiment of this invention. It is a flowchart for demonstrating operation | movement of the information processing apparatus concerning this embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In addition, in this specification and drawing, the description which overlaps by the same code | symbol may be attached | subjected about the component which has the same function.

(First embodiment)
First, a first embodiment of the present invention will be described.

  FIG. 1 is a block diagram showing a functional configuration of the information processing apparatus according to the first embodiment of the present invention. An information processing apparatus 100 illustrated in FIG. 1 includes, for example, a mobile phone, a smart phone, a game device, a PC (Personal Computer), a PDA (Personal Digital Assistant), a navigation device, a music playback device, a video processing device, and a server. Such as a device. Note that the PC includes a tablet PC and a notebook PC. The video processing device also includes a camera, a recorder, a player, and the like.

  The information processing apparatus 100 includes a line-of-sight information acquisition unit 11, an audio information acquisition unit 12, an operation signal generation unit 13, a control unit 14, and a storage unit 15.

  The line-of-sight information acquisition unit 11 acquires line-of-sight information indicating the user's line of sight. Here, the line of sight is a line connecting the center of the user's eyes and the object that the user is looking at. In the following description, the center of the eye, which is the starting point of the line of sight, is referred to as the viewpoint, and the point where the line of sight intersects the operation surface is referred to as the gaze point. The line-of-sight information includes, for example, the user's viewpoint and the line-of-sight direction from the viewpoint. Alternatively, the line-of-sight information may be gaze point position information. The line-of-sight information acquisition unit 11 outputs the acquired line-of-sight information to the operation signal generation unit 13.

  The sound information acquisition unit 12 acquires sound information indicating sound. The audio information includes volume information indicating the volume of the audio, for example. The audio information may include information indicating whether or not the volume is equal to or higher than a predetermined threshold (hereinafter referred to as a volume threshold). The audio information acquisition unit 12 outputs the acquired audio information to the operation signal generation unit 13.

  The operation signal generation unit 13 generates and outputs an operation signal indicating an input operation on the operation surface based on the line-of-sight information and the audio information. As a result, the information processing apparatus 100 executes processing according to the operation signal output from the operation signal generation unit 13.

  Specifically, the operation signal generation unit 13 specifies an input position on the operation surface based on the line-of-sight information, and determines whether the input position is selected based on the audio information. The operation signal generation unit 13 generates an operation signal based on the result of the determination and the specified input position.

  For example, the operation signal generation unit 13 sets the gazing point as the input position. The line-of-sight information includes information indicating the line of sight of the right eye and information indicating the line of sight of the left eye. At this time, the operation signal generation unit 13 specifies a first input position that is an input position corresponding to the right eye based on information indicating the line of sight of the right eye, and an input position corresponding to the left eye based on information indicating the line of sight of the left eye A second input position is specified. When the user's right eye gaze and left eye gaze intersect on the operation surface, the right eye gaze point and the left eye gaze point overlap each other, and the operation signal generation unit 13 performs one operation from the right eye gaze and the left eye gaze. Specify the input position. On the other hand, when the user's right eye gaze and left eye gaze do not intersect on the operation surface, the right eye gaze point and the left eye gaze point are different from each other, and the operation signal generation unit 13 is placed on the operation surface. Two input positions are specified.

  If the distance between the right eye point and the left eye point on the operation surface is equal to or smaller than a predetermined distance, the operation signal generator 13 specifies one input position from the right eye line and the left eye line. Also good. For example, the operation signal generation unit 13 specifies the position of the midpoint of the line segment connecting the right eye point and the left eye point on the operation surface as the input position.

  In addition, the operation signal generation unit 13 determines that the specified input position is selected when the sound volume is equal to or higher than the sound volume threshold. In this case, the user can perform a selection operation by blowing on a microphone (hereinafter abbreviated as a microphone) that acquires sound.

  Further, the operation signal generation unit 13 may determine whether or not an input position has been selected based on audio information other than the volume. For example, the operation signal generation unit 13 may determine whether an input position has been selected using voice recognition. In this case, the operation signal generator 13 determines that the input position has been selected when the input voice is analyzed to detect a predetermined word. Hereinafter, the description will be continued on the assumption that voice input is performed by blowing on the microphone.

  According to such a configuration, when the user turns his gaze on the operation surface, the position of the point on the operation surface specified by this gaze is handled as the input position (pointing position), and the user breathes into the microphone. It is determined that this input position has been selected.

  For example, in an input device such as a mouse or a touch input device, various input operations are used depending on the input position and the selection of the input position. For example, in a touch input device, a click operation, a long press operation, a drag operation, a pinch operation, and the like are used as an input operation. In addition, the mouse uses a pointing operation in addition to the input operation described above.

  Here, the click operation is an operation of touching a specific part on the operation surface and releasing it immediately thereafter. The long press operation is an operation of touching a specific part on the operation surface, waiting for a certain time, and releasing it. The drag operation is an operation of touching a specific part on the operation surface and moving and releasing the operation body while touching. The pinch operation is an operation of touching two points on the operation surface and changing the interval between the two touched points. The pointing operation is an operation of moving the mouse without pressing the button.

  Instead of touching the operation surface of the touch-type input unit or moving the mouse and pressing the button, the user performs the same operation as the input operation exemplified by moving the line of sight and blowing on the microphone. Can do.

  For example, when the user wants to perform a pointing operation, the user may turn his / her line of sight on the operation surface without performing an operation of blowing the microphone. In this case, the operation signal generation unit 13 specifies the input position based on the line-of-sight information and determines that the input position is not selected based on the audio information. In this case, the operation signal generated by the operation signal generation unit 13 is an operation signal corresponding to the pointing operation, and the information processing apparatus 100 performs the same processing as when the pointing operation is performed.

  Further, when the user wants to perform a click operation, the user only has to direct his / her line of sight to the position where he / she wants to click on the operation surface, and blow his / her breath on the microphone for a short time below the time threshold while fixing the line of sight. In this case, the operation signal generation unit 13 specifies the input position based on the line-of-sight information, and determines that the selection is canceled within a short period of time below the time threshold after the input position is selected once based on the audio information. To do. In this case, the operation signal generated by the operation signal generation unit 13 is an operation signal corresponding to the click operation, and the information processing apparatus 100 performs the same processing as when the click operation is performed.

  In addition, when the user wants to perform a long press operation, the user may direct the line of sight to the position where the user wants to long press on the operation surface, and keep blowing the line of sight for a time threshold or more while keeping the line of sight fixed. . At this time, the operation signal generation unit 13 specifies the input position based on the line-of-sight information, and after the input position is selected once based on the audio information and this selected state continues for a time threshold or more, It is determined that the selection has been canceled. In this case, the operation signal generated by the operation signal generation unit 13 is an operation signal corresponding to the long press operation, and the information processing apparatus 100 performs the same processing as when the long press operation is performed.

  Also, when the user wants to perform a drag operation, the user moves the line of sight in the direction in which he / she wants to drag while directing his / her line of sight on the operation surface to the position where the drag operation is to be started, It suffices to stop blowing when the line of sight is directed to the position where the drag operation is to be terminated. At this time, the operation signal generation unit 13 specifies the input position based on the line-of-sight information, and based on the audio information, the input position is selected once, and the selection is canceled after moving with the input position selected. to decide. In this case, the operation signal generated by the operation signal generation unit 13 is an operation signal corresponding to the drag operation, and the information processing apparatus 100 executes the same processing as when the drag operation is performed.

  In addition, when the user wants to perform a pinch operation, the user may perform an operation of changing the interval of the line of sight while blowing his / her breath on the microphone for a time threshold or more with the line of sight directed on the operation surface. Note that the user can narrow the interval of the line of sight by performing a so-called “more eye state” operation. In addition, the user can widen the line-of-sight interval by performing the operation of “external perspective”. At this time, the operation signal generation unit 13 identifies the input position based on the line-of-sight information, and detects that the distance between the two input positions changes when an operation of changing the line-of-sight interval is performed. In this case, the operation signal generated by the operation signal generation unit 13 is an operation signal corresponding to the pinch operation, and the information processing apparatus 100 performs the same processing as when the pinch operation is performed.

  The control unit 14 reads a program stored in the storage unit 15, executes the program, and operates according to an operation signal. The control unit 14 executes processing according to the operation signal output from the operation signal generation unit 13. For example, when the operation signal indicates an input position that is not selected, the control unit 14 displays a pointer at the input position indicated by the operation signal. Thereby, the user can recognize the position on the display screen currently indicated by the line of sight.

  The control unit 14 generates a display screen and causes the display device to display the display screen. At this time, the control unit 14 may be configured to be connected to an external display device or may be configured to include a display device. The display device is a display panel or a projector device that projects a display screen.

  The storage unit 15 is a storage medium that stores information. Here, the storage unit 15 is a nonvolatile memory such as a flash memory, an MRAM (Magnetoresistive Random Access Memory), an FeRAM (Ferroelectric Random Access Memory), a PRAM (Phase change Random Access Memory), and an EEPROM (Electronically Erasable and Programmable Read Only Memory). Or a magnetic recording medium such as an HDD (Hard Disk Drive). The storage unit 15 stores a program that defines the operation of the information processing apparatus 100, for example.

  Here, an example of a detailed configuration of the line-of-sight information acquisition unit 11 will be described. FIG. 2 is a block diagram illustrating a configuration example of the line-of-sight information acquisition unit according to the present embodiment. The line-of-sight information acquisition unit 11 includes a camera 21, a line-of-sight detection image storage memory 22, and a line-of-sight detection processing unit 23.

  The camera 21 is an imaging device that acquires a line-of-sight detection image obtained by imaging a user. The camera 21 acquires an image including the user's face, particularly the eyeball portion, as the line-of-sight detection image. FIG. 3 is an explanatory diagram illustrating an example of a line-of-sight detection image used for detecting a user's line of sight.

  The line-of-sight detection image storage memory 22 stores the line-of-sight detection image captured by the camera 21. The line-of-sight detection image storage memory 22 may be a storage device physically the same as the storage unit 15 illustrated in FIG. 1, for example, or may be a storage device separate from the storage unit 15.

  The line-of-sight detection processing unit 23 detects the line of sight of the user based on the line-of-sight detection image. For example, the line-of-sight detection processing unit 23 performs an edge detection process and a face detection process on the line-of-sight detection image to detect the line of sight. The edge detection process is a process for detecting a boundary between regions in an image. The change in luminance becomes large at the boundary of the region. For this reason, the line-of-sight detection processing unit 23 calculates the luminance Y for each pixel included in the line-of-sight detection image, and detects the boundary of the region by specifying the portion where the change in the luminance Y is large.

  Specifically, the line-of-sight detection processing unit 23 calculates the luminance for each pixel of the line-of-sight detection image using, for example, the following formula (1). Formula (1) is a conversion formula of ITU-T BT 601. Luminance is calculated from the intensity (R, G, B) of each color of red, green, and blue.

Y = (77 × R / 256) + (150 × G / 256) + (29 × B / 256) (1)
The line-of-sight detection processing unit 23 converts each pixel into a luminance signal, and then performs edge detection based on the absolute value of the difference between the luminance signal values of adjacent pixels. FIG. 4 is an explanatory diagram for explaining the edge detection processing result of the line-of-sight detection image. When edge detection is performed on the line-of-sight detection image of FIG. 3, a portion where the luminance change in the image is large, that is, a boundary portion of the region is detected as shown in FIG.

  The line-of-sight detection processing unit 23 detects a face portion in the line-of-sight detection image using the edge detection result and the face detection technique. For example, the line-of-sight detection processing unit 23 detects a downwardly convex part from the edge detection result, and further analyzes the color of the line-of-sight detection image. Can do.

  Further, when detecting the face portion in the line-of-sight detection image, the line-of-sight detection processing unit 23 detects eyes in the face portion. At this time, the line-of-sight detection processing unit 23 uses edge detection information. The line-of-sight detection processing unit 23 detects the position and orientation of the eyeball using the detected edge detection information inside the eye. The line-of-sight detection processing unit 23 detects the line of sight from the detected position and orientation of the eyeball.

  The detailed configuration example of the line-of-sight information acquisition unit 11 has been described above. Here, the line-of-sight information acquisition unit 11 includes the camera 21 that acquires the line-of-sight detection image, the line-of-sight detection image storage memory 22, and the line-of-sight detection processing unit 23 that analyzes the line-of-sight detection image. It is not limited to. For example, an external device connected to the information processing apparatus 100 may have some or all of the functions of the camera 21, the line-of-sight detection image storage memory 22, and the line-of-sight detection processing unit 23.

  An example of a detailed configuration of the voice information acquisition unit 12 will be described. FIG. 5 is a block diagram illustrating a configuration example of the audio information acquisition unit. The audio information acquisition unit 12 includes a microphone 24, an A / D (Analog / Digital) converter 25, a sound data storage memory 26, and a volume detection unit 27.

  The microphone 24 is a device that converts sound into an electrical signal. The microphone 24 converts the sound into an analog signal and outputs it to the A / D converter 25.

  The A / D converter 25 converts the analog signal of the sound output from the microphone 24 into a digital signal. The A / D converter 25 converts an analog signal into a digital signal using, for example, ΣΔ modulation. Further, the A / D converter 25 samples an analog signal 8000 times per second, for example, and acquires a digital signal indicated by a digital quantity.

  The sound data storage memory 26 is a storage unit that stores sound data converted into a digital signal by the A / D converter 25. The sound data storage memory 26 may be a storage device physically the same as the storage unit 15 shown in FIG. 1, for example, or may be a separate storage device from the storage unit 15.

  The sound volume detector 27 detects the sound volume from the sound data stored in the sound data storage memory 26. More specifically, the volume detector 27 detects the volume based on the absolute value of the digital value of the sound data. The volume detection unit 27 outputs the absolute value of the digital quantity as volume information to the operation signal generation unit 13.

  The sound volume detection unit 27 may determine whether the sound volume is equal to or higher than the sound volume threshold value and output information indicating whether the sound volume is equal to or higher than the sound volume threshold value to the operation signal generation unit 13 as sound volume information. Here, the volume detector 27 determines that the volume of this sound is equal to or greater than the volume threshold when, for example, the number of samples whose digital value absolute value is equal to or greater than the volume threshold is equal to or greater than a certain ratio.

  Heretofore, the functional configuration of the information processing apparatus 100 has been described. The hardware configuration for realizing the functions shown here can take various forms. For example, in the above embodiment, the information processing apparatus 100 includes the camera 21 and the microphone 24, but the present invention is not limited to such an example. For example, the camera 21 and the microphone 24 may be provided in an external device such as a headset, and the information processing apparatus 100 may acquire a line-of-sight detection image and sound data from the external device. The external device may have a function of analyzing the line-of-sight detection image and the sound data.

  Next, an example of determination processing in which the control unit 14 determines processing to be executed by an input operation indicated by the operation signal generated by the operation signal generation unit 13 will be described. FIG. 6 is a flowchart for explaining an example of a determination process in which the control unit according to the present embodiment determines a process to be executed by an input operation indicated by the operation signal. Note that the determination process of FIG. 6 is executed in a state where the input position is specified based on the line of sight.

  First, the control unit 14 determines whether or not the input position is selected based on the operation signal (step S100). When the input position is not selected, the control unit 14 executes a process associated with the pointing operation (step S105).

  On the other hand, when the input position is selected, the control unit 14 determines whether or not the next selected state continues for a time threshold or more (step S110). When the selected state does not continue for the time threshold or more, the control unit 14 executes a process associated with the click operation (step S115).

  On the other hand, when the selected state continues for the time threshold or more, the control unit 14 determines whether or not the input position has moved (step S120). If the input position has not moved, the control unit 14 executes a process associated with the long press operation (step S125).

  On the other hand, when the input position has moved, the control unit 14 next determines whether or not the interval between the left and right input positions has changed (step S130). When the interval between the left and right input positions has not changed, the control unit 14 executes a process associated with the drag operation (step S135). On the other hand, when the interval between the left and right input positions changes, the control unit 14 executes a process associated with the pinch operation (step S140).

  As described above, according to the present embodiment, an operation signal indicating an input operation on the operation surface can be generated based on line-of-sight information and audio information. For this reason, the user can realize an input operation using something other than the hand. In addition, various input operations are possible as compared with the case where only the line of sight or only the voice is used. Therefore, for example, even in a situation where it is difficult to use a hand, the user can perform an operation, and thus an input operation with excellent operability can be realized.

  In the present embodiment, the operation signal generation unit 13 specifies the input position on the operation surface based on the line-of-sight information, determines whether the input position is selected based on the audio information, and determines the result of the determination and An operation signal is generated based on the input position. For this reason, the user can designate a position on the operation surface by inputting a line of sight, and can operate whether or not to select the designated position by inputting voice. An input operation performed by selecting an input position on the operation surface and moving the selected input position is widely used by a mouse or a touch sensor. According to said structure, the user can perform the familiar operation which had been performed with the mouse | mouth, the touch sensor, etc. using a visual line and an audio | voice.

  In the present embodiment, the sound information includes sound volume information indicating the sound volume, and the operation signal generation unit 13 determines whether or not an input position has been selected based on the sound volume information. Specifically, the operation signal generation unit 13 determines that the input position has been selected when the volume is equal to or greater than the volume threshold. By using the sound volume to determine whether or not the input position has been selected, the information processing apparatus 100 can perform an input operation, for example, by blowing on a microphone that acquires sound. The operation signal generator 13 may determine whether or not the input position is selected using, for example, voice recognition based on voice information other than the volume. There is a dependency, and the user may feel resistance to uttering a voice toward a device rather than a person. On the other hand, when sound is input by blowing on the microphone using the sound volume, there is an advantage that there is no language dependence and the above resistance can be reduced.

  Further, in the present embodiment, the operation signal generation unit 13 specifies the first input position that is the input position corresponding to the right eye based on the information indicating the right eye line of sight included in the line of sight information, and indicates the left eye line of sight The second input position, which is the input position corresponding to the left eye, is specified based on the above. When the right-eye gazing point and the left-eye gazing point coincide with each other, the first input position and the second input position overlap with each other, and can be handled as one input position. On the other hand, when the user performs a so-called “more eyes” or “external perspective” state, the first input position is different from the second input position. Therefore, in this case, the user can input two positions on the operation surface.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.

  FIG. 7 is a block diagram showing a functional configuration of the information processing apparatus according to the second embodiment of the present invention. An information processing apparatus 200 illustrated in FIG. 7 includes a line-of-sight information acquisition unit 11, an audio information acquisition unit 12, an operation signal generation unit 13, a control unit 14, a storage unit 15, a touch input unit 16, and an operation mode. And a setting unit 17. That is, the information processing apparatus 200 further includes a touch input unit 16 and an operation mode setting unit 17 in addition to the configuration of the information processing apparatus 100 according to the first embodiment. Hereinafter, the description of the same configuration as the information processing apparatus 100 according to the first embodiment will be omitted, and the difference will be mainly described.

  The touch-type input unit 16 has an operation surface and detects the position of the operation body that is in contact with or close to the operation surface. The touch input unit 16 outputs the detected position information of the operating tool to the operation signal generation unit 13.

  The operation mode setting unit 17 sets an operation mode in the operation signal generation unit 13. The operation mode setting unit 17 generates a first operation mode based on the position detected by the touch-type input unit 16, and a second operation mode generates a manipulation signal based on the line-of-sight information and audio information. , One of the operation modes is set in the operation signal generator 13. The operation mode setting unit 17 stops the supply of power to the touch input unit 16 when setting the second operation mode in the operation signal generation unit 13. The operation mode setting unit 17 detects an operation mode setting operation input by the user, for example, and sets the operation mode according to the operation mode setting operation.

  Here, an operation example of the operation mode setting unit 17 of the information processing apparatus 200 according to the second embodiment of the present invention will be described. FIG. 8 is a flowchart for explaining an operation example of the operation mode setting unit according to the present embodiment.

  First, the operation mode setting unit 17 detects an operation mode setting operation based on the operation signal supplied from the operation signal generation unit 13 (step S200). Then, the operation mode setting unit 17 sets the operation mode according to the operation mode setting operation (step S205).

  Subsequently, the operation mode setting unit 17 determines whether or not the set operation mode is the second operation mode (step S210). When the operation mode set here is the second operation mode, the operation mode setting unit 17 stops the supply of power to the touch input unit 16 (step S215).

  As described above, in the present embodiment, the information processing apparatus 200 includes the touch-type input unit 16 that detects a position in contact with or close to the operation surface, and based on the position detected by the touch-type input unit 16. The operation signal generation unit 13 is set to one of the first operation mode for generating the operation signal and the second operation mode for generating the operation signal based on the line-of-sight information and the audio information. A mode setting unit 17 is further included. The operation mode setting unit 17 stops supplying power to the touch input unit 16 when setting the operation signal generation unit 13 to the second operation mode. Thereby, when the information processing apparatus 200 operates in the second operation mode in which the operation signal is generated based on the line-of-sight information and the sound information, the power consumption corresponding to the operation power of the touch input unit 16 is reduced. Can do.

(Third embodiment)
Next, a third embodiment of the present invention will be described.

  FIG. 9 is a block diagram showing a functional configuration of an information processing apparatus according to the third embodiment of the present invention. An information processing apparatus 300 illustrated in FIG. 9 includes a line-of-sight information acquisition unit 31, an audio information acquisition unit 32, and an operation signal generation unit 33.

  The line-of-sight information acquisition unit 31 acquires line-of-sight information indicating the user's line of sight.

  The sound information acquisition unit 32 acquires sound information indicating sound.

  The operation signal generation unit 33 generates an operation signal indicating an input operation on the operation surface based on the line-of-sight information and the sound information.

  Next, the operation of the information processing apparatus 300 according to the present embodiment will be described. FIG. 10 is a flowchart for explaining the operation of the information processing apparatus according to the present embodiment.

  The line-of-sight information acquisition unit 31 acquires line-of-sight information (step S300). The voice information acquisition unit 32 acquires voice information (step S305). Then, the operation signal generator 33 generates an operation signal based on the line-of-sight information and the audio information (step 310).

  Also in the present embodiment, an operation signal indicating an input operation on the operation surface can be generated based on the line-of-sight information and audio information. For this reason, the user can realize an input operation using something other than the hand. In addition, various input operations are possible as compared with the case where only the line of sight or only the voice is used. Therefore, for example, even in a situation where it is difficult to use a hand, the user can perform an operation, and thus an input operation with excellent operability can be realized.

  The preferred embodiments of the present invention have been described above. However, the configuration shown in each embodiment described above is merely an example, and the present invention is not limited to the configuration.

  For example, the functions of the information processing apparatuses 100, 200, and 300 are recorded on a computer-readable recording medium and a program recorded on the recording medium is read by the computer. It may be realized by executing.

  Moreover, the input operation described in the above embodiment is an example, and various input operations using an input position and information indicating whether or not the input state is selected may be used. For example, an input operation similar to a double click operation, a flick operation, or the like may be used.

  Moreover, although the example of input operation which does not use a hand was described in the said embodiment, this invention is not limited to this example. For example, the operation using the hand and the operation based on the line-of-sight information and the audio information described in the above embodiment can be used in combination.

100, 200, 300 Information processing apparatus 11, 31 Line-of-sight information acquisition unit 12, 32 Audio information acquisition unit 13, 33 Operation signal generation unit 14 Control unit 15 Storage unit 16 Touch type input unit 17 Operation mode setting unit

Claims (8)

A line-of-sight information acquisition unit for acquiring line-of-sight information indicating the user's line of sight;
A voice information acquisition unit that acquires voice information indicating voice;
An operation signal generation unit that generates an operation signal indicating an input operation on the operation surface based on the line-of-sight information and the audio information;
An information processing apparatus comprising: The operation signal generation unit specifies an input position on the operation surface based on the line-of-sight information, determines whether the input position is selected based on the audio information, and determines the result of the determination and the input Generating the operation signal based on the position;
The information processing apparatus according to claim 1. The audio information includes volume information indicating a volume of the audio,
The operation signal generator determines whether the input position is selected based on the volume information;
The information processing apparatus according to claim 2. The operation signal generation unit determines that the input position is selected when the volume is equal to or higher than a volume threshold.
The information processing apparatus according to claim 3. The operation signal generation unit, based on each of information indicating a right eye line of sight and information indicating a left eye line of sight included in the line of sight information, as the input position, the first input position corresponding to the right eye and the Identify each second input position corresponding to the left eye,
The operation signal generation unit generates the operation signal based on the first input position and the second input position.
The information processing apparatus according to any one of claims 2 to 4. A touch-type input unit that detects a position in contact with or close to the operation surface;
A first operation mode for generating the operation signal based on the position detected by the touch input unit, and a second operation signal for generating the operation signal based on the line-of-sight information and the audio information. An operation mode setting unit for setting any one of the operation modes,
Further comprising
The operation mode setting unit stops supplying power to the touch input unit when setting the second operation mode in the operation signal generation unit.
The information processing apparatus according to any one of claims 1 to 5. Get line-of-sight information showing the user's line of sight
Acquire audio information indicating the audio,
Based on the line-of-sight information and the audio information, an operation signal indicating an input operation on the operation surface is generated.
Operation signal generation method. A procedure for obtaining gaze information indicating a user's gaze;
A procedure for obtaining audio information indicating audio;
A procedure for generating an operation signal indicating an input operation to the operation surface based on the line-of-sight information and the audio information;
A program that causes a computer to execute.