JP6131540B2 - Tablet terminal, operation reception method and operation reception program - Google Patents

Description

  The present invention relates to a tablet terminal and the like.

  Smartphones, mobile phones, tablet terminals such as PHS (Personal Handyphone System) and PDA (Personal Digital Assistants) are widely used. Since the tablet terminal is small and light, the user can use the tablet terminal anywhere.

  When the user operates the tablet terminal, the user operates by touching various items displayed on the touch panel of the tablet terminal with a finger. For example, the user performs a touch operation to select hypertext on a Web page displayed on the touch panel, enlarge an image, play or stop a moving image, and the like.

“MobiGaze, Development of a Gaze Interface for Handheld Mobile Devices”, April 12-13 2010

  However, the above-described conventional technique has a problem that it cannot be comfortably operated when the touch panel is operated with one hand.

  For example, when a user is holding a train strap or holding a bag, umbrella, etc. with one hand, the touch panel of the tablet terminal is operated with one hand. When operating the touch panel under such circumstances, the reach of the user's finger is limited, and the touch panel cannot be operated comfortably.

  The disclosed technology has been made in view of the above, and an object thereof is to provide a tablet terminal, an operation reception method, and an operation reception program that can be comfortably operated even when the touch panel is operated with one hand. .

  The disclosed tablet terminal includes an area specifying unit, an operation determination unit, and a display control unit. The area specifying unit specifies an area where a touch operation is performed on the touch panel. The operation determining unit determines the content of the touch operation in the area specified by the area specifying unit. The display control unit generates an option corresponding to the content of the touch operation determined by the operation determination unit, and displays the generated option in a predetermined area of the touch panel.

  According to the tablet terminal of an indication, even when it operates with one hand, there exists an effect that a touch panel can be operated comfortably.

FIG. 1 is a diagram illustrating the configuration of the tablet terminal according to the first embodiment. FIG. 2 is a diagram for explaining each region in the screen of the touch panel. FIG. 3 is a diagram for explaining the target area. FIG. 4 is a diagram for explaining the predetermined area. FIG. 5 is a diagram (1) illustrating an example of the menu image. FIG. 6 is a diagram (2) illustrating an example of the menu image. FIG. 7 is a diagram (3) illustrating an example of the menu image. FIG. 8 is a diagram (1) illustrating an example of correspondence information. FIG. 9 is a diagram (2) illustrating an example of correspondence information. FIG. 10 is a diagram (3) illustrating an example of correspondence information. FIG. 11 is a diagram (1) illustrating a screen image of the tablet terminal according to the first embodiment. FIG. 12 is a diagram (2) illustrating a screen image of the tablet terminal according to the first embodiment. FIG. 13 is a diagram (3) illustrating a screen image of the tablet terminal according to the first embodiment. FIG. 14 is a flowchart (1) illustrating the processing procedure of the tablet terminal according to the first embodiment. FIG. 15 is a flowchart (2) illustrating the processing procedure of the tablet terminal according to the first embodiment. FIG. 16 is a flowchart (3) illustrating the processing procedure of the tablet terminal according to the first embodiment. FIG. 17 is a diagram illustrating the configuration of the tablet terminal according to the second embodiment. FIG. 18 is a diagram illustrating a screen image of the tablet terminal according to the second embodiment. FIG. 19 is a flowchart illustrating the processing procedure of the tablet terminal according to the second embodiment. FIG. 20 is a diagram (1) illustrating an example of another menu image. FIG. 21 is a diagram (2) illustrating an example of another menu image. FIG. 22 is a schematic diagram illustrating an example of a computer that executes an operation reception program according to the embodiment.

  Hereinafter, embodiments of a tablet terminal, an operation reception method, and an operation reception program disclosed in the present application will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

  A configuration of the tablet terminal according to the first embodiment will be described. FIG. 1 is a diagram illustrating the configuration of the tablet terminal according to the first embodiment. As illustrated in FIG. 1, the tablet terminal 100 includes a camera 110, a light source 120, a communication unit 130, a touch panel 140, a storage unit 150, and a control unit 160. Other configurations are the same as those of a known tablet terminal.

  The camera 110 is an imaging device using a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS), or the like. The camera 110 captures an image of a subject included in the imaging range, and outputs image data of the captured subject to the line-of-sight detection unit 161.

  The camera 110 controls the light source 120 at the timing of taking an image and irradiates the subject with infrared rays. When the subject photographed by the camera 110 includes a user's eyeball, the reflection of light on the retina is captured by the camera 110, and an image in which the pupil part of the eyeball appears brighter than the other parts is obtained. The camera 110 may be a camera installed from the time of shipment of the tablet terminal 100, or may be an externally connected digital camera or the like.

  The light source 120 is a device that emits infrared light in response to a control command of the camera 110. The communication unit 130 is a processing unit that connects to a network and executes data communication with other devices. For example, the communication unit 130 corresponds to a wireless communication device. A control unit 160 described later accesses a server on the network via the communication unit 130.

  The touch panel 140 is an interface having two functions of display and input. Touch panel 140 displays information output from control unit 160. In addition, when the touch operation is accepted, the touch panel 140 outputs information on the touched screen position to the control unit 160.

  In the first embodiment, a line-of-sight detection area, a target area, and a default area are defined in the screen of the touch panel 140 for convenience of explanation. FIG. 2 is a diagram for explaining each region in the screen of the touch panel. As illustrated in FIG. 2, the touch panel 140 includes a line-of-sight detection area 141, a target area 142, and a predetermined area 143.

  For example, the line-of-sight detection area 141 is an area corresponding to the display screen of the touch panel 140. The target area 142 is an area including the line-of-sight position 10a. The line-of-sight position 10a is a line-of-sight position on the user's screen detected by the line-of-sight detection unit 161 described later. The default area 143 is an area where the thumb 10b holding the tablet terminal 100 reaches when the user operates the tablet terminal 100 with one hand.

  The target area 142 will be described. FIG. 3 is a diagram for explaining the target area. As shown in FIG. 3, the target area 142 is defined by a width ll and a height ls, and a centroid of a rectangle with the width ll and the height ls is a centroid position Gd (xd, yd). The center of gravity position Gd of the target area 142 is adjusted to a position that overlaps the user's line-of-sight position 10a by the area specifying unit 162 described later, whereby the position of the target area 142 is also moved. It is assumed that the values of the width ll and the height ls are set in advance.

  The default area 143 will be described. FIG. 4 is a diagram for explaining the predetermined area. As shown in FIG. 4, the predetermined area 143 is defined by a centroid position Gd (xd, yd) where the centroid of the predetermined area 143 exists, a width ll, and a height ls. In the predetermined area 143, a shape and a size equivalent to or smaller than the range 20 that the user reaches with the thumb 10b of the hand holding the tablet terminal 100 holding the tablet terminal 100 are defined. When the vertical direction of the image displayed on the screen of the touch panel 140 is determined, the gravity center position Gd (xd, yd) so that the lower left corner where the screen is displayed and the lower left vertex of the default area 143 coincide with each other. ).

  Returning to the description of FIG. The storage unit 150 is a storage device that stores menu image information 151, correspondence information 152, display information 153, and rectangle definition information 154. For example, the storage unit 150 corresponds to a semiconductor memory device such as a random access memory (RAM), a read only memory (ROM), or a flash memory, or a storage device such as a hard disk or an optical disk.

  The menu image information 151 includes information on various menu images for performing various operations. The menu image is displayed in the default area 143 of the touch panel 140, and the user performs a touch operation on the menu image. Here, an example of each menu image will be described. 5-7 is a figure which shows an example of a menu image.

  The menu image 30 shown in FIG. 5 is for adjusting the size of the image. For example, when the user slides the arrow 30a of the menu image 30 from the left side to the right side, an operation for enlarging the corresponding image or text character is performed.

  The menu image 31 shown in FIG. 6 has a plurality of selection buttons 31a, 31b, and 31c. As will be described later, the selection buttons 31a to 31c are associated with the selection items displayed on the screen on the touch panel 140, respectively, and the corresponding selection items are selected when the user touches the selection buttons 31a to 31c. It becomes operation to do. In the case of a Web page as an example, the selection item corresponds to hypertext or the like.

  A menu image 32 shown in FIG. 7 is used for operating a moving image. For example, the menu image 32 has a play button 32a, a stop button 32b, and a sound button 32c. For example, when the user performs a touch operation on the play button 32a, an operation of playing the corresponding moving image is performed. When the user touches the stop button 32b, the corresponding moving image is stopped. When the user performs a touch operation on the sound button 32c, the operation switches the sound of the moving image between sound and silence.

  The menu images included in the menu image information 151 are not limited to the menu images shown in FIGS. Further, the contents of the menu images shown in FIGS. 5 to 7 are merely examples, and may be changed as appropriate.

  Returning to the description of FIG. The correspondence relationship information 152 is information for associating the touch operation on the menu image shown in FIGS. 5 to 7 with the operation on the item on the touch panel 140. The correspondence information 152 includes a plurality of pieces of correspondence information corresponding to the menu image. For example, let the correspondence information corresponding to the menu image 30 be the correspondence information 152a. The correspondence information corresponding to the menu image 31 is assumed to be correspondence information 152b. The correspondence information corresponding to the menu image 32 is assumed to be correspondence information 152c. 8-10 is a figure which shows an example of correspondence information.

  FIG. 8 shows an example of the correspondence information 152 a corresponding to the menu image 30. As shown in FIG. 8, this correspondence information 152a associates the x-coordinate with the image magnification. The x coordinate corresponds to the x coordinate of the arrow 30 a of the menu image 30. The image enlargement rate corresponds to the enlargement rate when enlarging the corresponding image or text characters. For example, when the x coordinate of the arrow 30a is “0”, the image enlargement ratio is “10%”. When the x coordinate of the arrow 30a is “xd”, the image enlargement ratio is “100%”. When the x coordinate of the arrow 30a is “xm”, the image enlargement ratio is “200%”.

  FIG. 9 shows an example of the correspondence information 152b corresponding to the menu image 31. As shown in FIG. 9, the correspondence information 152b associates button identification information, operation details, and link coordinates. The button identification information is information for identifying each selection button included in the menu image 31. The operation content is information indicating the operation content when the selection button is touched. The link coordinates indicate the coordinates of the selection item on the touch panel 140. For example, when the selection button 31a is touched, the selection item of the link coordinates “x1, y1” on the touch panel 140 is “selected”.

  FIG. 10 shows an example of the correspondence information 152 c corresponding to the menu image 32. As shown in FIG. 10, the correspondence information 152c associates button identification information, operation details, and a linked moving image area. The button identification information is information for identifying each button included in the menu image 32. The operation content is information indicating the operation content when the selection button is touched. The linked moving image area indicates an area on the touch panel 140 of the moving image operated by the menu image 32. The linked moving image area includes, for example, upper left coordinates and lower right coordinates of the moving image. When the playback button 32a is touched, the video in the video area “(x4, y4), (x5, y5)” on the touch panel 140 is “played”.

  Returning to the description of FIG. The display information 153 is screen information displayed on the touch panel 140. For example, when Web page information is displayed on the touch panel 140, the display information 153 includes the source code of the Web page.

  The rectangle definition information 154 includes information on the range of the line-of-sight detection area 141 illustrated in FIG. 2, the length of each side of the target area 142, and the length of each side of the default area 143.

  The control unit 160 includes a line-of-sight detection unit 161, a region specifying unit 162, an operation determination unit 163, a touch determination unit 164, and a display control unit 165. For example, the control unit 160 corresponds to an integrated device such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). Moreover, the control part 160 respond | corresponds to electronic circuits, such as CPU and MPU (Micro Processing Unit), for example.

  The line-of-sight detection unit 161 is a processing unit that detects the line-of-sight position on the touch panel 140. The line-of-sight detection unit 161 outputs information on the line-of-sight position to the region specifying unit 162. For example, the line-of-sight detection unit 161 applies an algorithm such as a corneal reflection method to the subject image output by the camera 110, and detects the position of the viewpoint indicated by the direction of the line of sight from the center position of the pupil of the eyeball. In this embodiment, the position of the viewpoint is the line-of-sight position.

  Here, the case where the line-of-sight position is detected using the cornea reflection method has been described, but the line-of-sight position may be detected using another method. For example, the tablet terminal 100 divides the screen of the touch panel 140, learns the shape of the eye that sees the divided area, and executes template matching with the shape of the eye extracted from the image input from the camera 110. The line-of-sight position may be detected. The tablet terminal 100 may acquire a position of the line of sight detected by the headset by causing the user to wear a headset that detects the position of the line of sight.

  The line-of-sight detection unit 161 detects the line-of-sight position, and determines whether or not the detected line-of-sight position is included in the line-of-sight detection area 141 based on the rectangle definition information 154. The line-of-sight detection unit 161 outputs information on the line-of-sight position to the region specifying unit 162 when the line-of-sight position is included in the line-of-sight detection region 141.

  The region specifying unit 162 is a processing unit that specifies a region where a touch operation is performed. In the following, an example of processing of the region specifying unit 162 will be described. The region specifying unit 162 acquires the information on the line-of-sight position from the line-of-sight detection unit 161, and calculates the average of the line-of-sight position every predetermined time. The area specifying unit 162 specifies the range of the target area 142 shown in FIG. 2 by using the average line-of-sight position as the barycentric coordinates of the target area. The area specifying unit 162 outputs information on the range of the specified target area 142 to the operation determining unit 163. The information on the range of the target area 142 is, for example, coordinates on the touch panel 140 of the target area 142.

  The operation determination unit 163 is a processing unit that determines the content of the touch operation in the target area 142. The operation determination unit 163 outputs the content of the determined touch operation to the display control unit 165.

  Hereinafter, an example of processing of the operation determination unit 163 will be described. The operation determination unit 163 inquires about an element included in the target area 142 using DOM (Document Object Model). DOM is an interface for accessing each element of xml and html. For example, the operation determination unit 163 uses JavaScript (registered trademark) elementFromPoint (x, y). In the present embodiment, the operation determination unit 163 acquires the source code corresponding to the target area 142 from the display information 153 using DOM.

  When the element included in the target area 142 includes a tag for identifying an image or a character, the operation determination unit 163 determines that an operation for adjusting the size is performed. For example, a tag for identifying an image is <imag>, and a tag for identifying a character is <p>. Other tags for identifying images and characters are defined in the HTML reference and the like, and thus description thereof is omitted.

  If it is determined that an operation for adjusting the size is to be performed, the operation determining unit 163 outputs information indicating that the operation for adjusting the size is performed and the range of the target area 142 to the display control unit 165.

  If the element included in the target area 142 includes a tag that identifies the link destination, the operation determination unit 163 determines that an operation for selecting the link destination is performed. In addition, the operation determination unit 163 adds up the number of links. For example, the tag for identifying the link destination is <a href>. Other tags for identifying link destinations are defined in the HTML reference and the like, and thus the description thereof is omitted.

  If the operation determining unit 163 determines to perform an operation of selecting a link destination, the display control unit 165 indicates that the link destination is selected, the number of links, and the coordinates of the item to be selected on the screen. Output to.

  If the element included in the target area 142 includes a tag for identifying a moving image, the operation determining unit 163 determines that the moving image is to be operated. For example, a tag for identifying a moving image is <video>. Other tags for identifying moving images are defined in the HTML reference and the like, and thus description thereof is omitted.

  When the operation determination unit 163 determines to perform the operation of the moving image, the operation determination unit 163 outputs information indicating that the operation of the moving image is performed and the range of the target area 142 to the display control unit 165.

  The touch determination unit 164 is a processing unit that determines whether or not a touch operation has been performed on the touch panel 140. The touch determination unit 164 outputs a determination result on whether or not a touch operation has been performed to the display control unit 165.

  The display control unit 165 is a processing unit that generates options corresponding to the determination result of the operation determination unit 163 and displays the options in the default area 143 of the touch panel 140. Hereinafter, an example of processing of the display control unit 165 will be described. Note that the display control unit 165 executes the following processing using the information indicating that the touch operation has been performed from the touch determination unit 164 as a trigger.

  A process when the display control unit 165 receives an operation of adjusting the size from the operation determination unit 163 and information on the range of the target area 142 will be described. The display control unit 165 acquires the menu image 30 for performing the enlargement operation illustrated in FIG. 5 from the menu image information 151. In addition, the display control unit 165 acquires the correspondence information 152a corresponding to the menu image 30 from the correspondence information 152, and associates the operation of the arrow 30a of the menu image 30 with the content of the operation.

  The display control unit 165 displays the menu image 30 in the default area 143 after associating the contents of the operation based on the menu image 30 and the correspondence information 152a. When the display control unit 165 receives a touch operation on the arrow 30a of the menu image 30, the display control unit 165 determines the image enlargement ratio based on the coordinates of the arrow 30a and the correspondence information 152a. The display control unit 165 enlarges the image or characters included in the target area 142 according to the image enlargement rate.

  Next, processing when the display control unit 165 receives from the operation determination unit 163 selection of a link destination, the number of links, and coordinates on the screen of an item to be selected will be described. The display control unit 165 acquires the menu image 31 including the selection button illustrated in FIG. 6 from the menu screen information 151. Further, the display control unit 165 generates the correspondence information 152 shown in FIG. 9 by associating each selection button of the menu image 31 with the coordinates on the screen of the item to be selected. In addition, the contents of each operation are set to select the selection item. In addition, the display control part 165 shall increase the selection button of the menu image 31 shown in FIG. 6 according to the number of links.

  After generating the correspondence information 152, the display control unit 165 displays the menu image 31 in the default area 143. When the display control unit 165 receives a touch operation on the selection button of the menu image 31, the display control unit 165 compares the button identification information of the selection button that has received the touch operation with the correspondence information 152b to determine the link coordinates. The display control unit 165 selects a link destination item corresponding to the determined link destination coordinates and causes the touch panel 140 to display link destination information. For example, when the link destination item is hypertext, the display control unit 165 acquires the link destination information indicated in the hypertext via the network, and causes the touch panel 140 to display the acquired information.

  Next, a process when the display control unit 165 receives a moving image operation and information on the range of the target area 142 from the operation determination unit 163 will be described. The display control unit 165 acquires the menu image 32 for operating the moving image shown in FIG. 7 from the menu image information 151. In addition, the display control unit 165 acquires the correspondence information 152c corresponding to the menu image 32 from the correspondence information 152, and the linked moving image corresponding to the play button 32a, the stop button 32b, and the sound button 32c of the menu image 32. The area is set to the coordinates of the range of the target area 142.

  The display control unit 165 causes the menu image 32 to be displayed in the default area 143 after associating the menu image 32 with the correspondence information 152c. When the display control unit 165 receives a touch operation on the button of the menu image 32, the display control unit 165 identifies the operation content based on the button identification information and the correspondence information 152c, and executes a process according to the operation content. . The display control unit 165 specifies the coordinates of the moving image to be processed based on the linked moving image area.

  For example, when a touch operation is performed on the reproduction button 32a, the display control unit 165 reproduces the moving image in the corresponding area. When a touch operation is performed on the stop button 32b, the display control unit 165 stops the moving image in the corresponding area. When a touch operation is performed on the sound button 32c, the display control device 165 switches between sound and silence of the corresponding moving image.

  Note that the display control unit 165 deletes the menu image when the touch operation for the corresponding menu image is not received within a predetermined time after the menu image 30, 31 or 32 is displayed in the default area 143.

  Next, a screen image of the tablet terminal 100 according to the first embodiment will be described. 11 to 13 are diagrams illustrating screen images of the tablet terminal according to the first embodiment.

  FIG. 11 is a screen image when an operation for adjusting the size is performed. As shown in FIG. 11, the tablet terminal 100 detects the line-of-sight position 10a and sets the target area 142 (step S10). Using the DOM, the tablet terminal 100 determines that the target area 142 is the image 20a (step S11).

  The tablet terminal 100 displays the menu image 30 for performing the enlargement operation in the default area 143 (step S12). The user slides the arrow on the menu image 30 (step S13). The tablet terminal 100 compares the coordinates of the arrow with the correspondence relationship information 152a, determines the image enlargement rate, and enlarges the image 20 in accordance with the image enlargement rate (step S14).

  FIG. 12 is a screen image when a link destination is selected. As illustrated in FIG. 12, the tablet terminal 100 detects the line-of-sight position 10a and sets the target area 142 (step S20). The tablet terminal 100 specifies, for example, the link destinations 21, 22, and 23 from the target area 142 using DOM (step S21).

  The tablet terminal 100 displays a menu image 31 for selecting a link destination in the default area 143. For example, when there are three link destinations, the tablet terminal 100 displays the menu image 31 having the three selection buttons 31a, 31b, and 31c in the default area 143 (step S22). For example, the selection buttons 31a, 31b, and 31c correspond to the link destinations 21, 22, and 23.

  For example, the user performs a touch operation on the selection button 31a of the menu image 31 (step S23). Then, the tablet terminal 100 determines that the link destination 21 corresponding to the selection button 31a has been selected, and displays information linked to the link destination 21 (step S24).

  FIG. 13 is a screen image when operating a moving image. As illustrated in FIG. 13, the tablet terminal 100 detects the line-of-sight position 10a and sets the target area 142 (step S30). The tablet terminal 100 determines that a moving image exists in the target area 142 using DOM. The tablet terminal 100 may further specify the moving image area 142a (step S31).

  The tablet terminal 100 displays the menu image 32 for operating the moving image in the default area 143 (step S32). For example, the menu image 32 includes a play button 32a, a stop button 32b, and a sound button 32c.

  For example, the user performs a touch operation on the playback button 32a (step S33). When the playback button 32a is touched, the tablet terminal 100 plays back a moving image (step S34). Note that the tablet terminal 100 stops the moving image when the stop button 32b is touched. When the sound button 32c is touched, the tablet terminal 100 switches between moving sound and silent.

  Next, a processing procedure of the tablet terminal 100 according to the first embodiment will be described. 14 to 16 are flowcharts illustrating the processing procedure of the tablet terminal according to the first embodiment. Here, for convenience of explanation, the description is divided into the case of performing an operation of adjusting the size, the operation of selecting a link destination, and the operation of a moving image.

  FIG. 14 is a flowchart in the case of performing an operation of adjusting the size. The process illustrated in FIG. 14 is executed, for example, when the image data is acquired from the camera 110. As illustrated in FIG. 14, the tablet terminal 100 detects the line-of-sight position (step S101), and determines whether the line-of-sight position is within the screen and the touch panel 140 is tapped (step S102). If the line-of-sight position is not within the screen or if the touch panel 140 is not tapped (No at Step S102), the tablet terminal 100 proceeds to Step S101.

  On the other hand, when the line-of-sight position is within the screen and the touch panel 140 is tapped (Yes in step S102), the tablet terminal 100 sets the target area 142 using the line-of-sight position as the center of gravity (step S103). The tablet terminal 100 analyzes the DOM information in the HTML in the target area 142 and specifies the image in the target area 142 (step S104).

  The tablet terminal 100 determines the arrangement position of the menu image 30 (step S105), and acquires correspondence information 152a in which the position of the menu image 30 is associated with the content of the operation (step S106). The tablet terminal 100 displays the menu image 30 in the default area 143 (step S107).

  The tablet terminal 100 determines whether a touch operation has been received within a specified time (step S108). If the tablet terminal 100 does not accept the touch operation within the predetermined time (step S108, No), the menu image 30 is deleted (step S109), and the process proceeds to step S101.

  On the other hand, when the tablet terminal 100 receives a touch operation within a predetermined time (step S108, Yes), the tablet terminal 100 specifies the image enlargement ratio based on the touch operation and the correspondence information 152a (step S110). The tablet terminal 100 executes enlargement (step S111).

  FIG. 15 is a flowchart when an operation for selecting a link destination is performed. The process illustrated in FIG. 15 is executed when the image data is acquired from the camera 110, for example. As illustrated in FIG. 15, the tablet terminal 100 detects the line-of-sight position (step S201), and determines whether the line-of-sight position is within the screen and the touch panel 140 is tapped (step S202). When the line-of-sight position is not within the screen or when the touch panel 140 is not tapped (No at Step S202), the tablet terminal 100 proceeds to Step S201.

  On the other hand, when the line-of-sight position is within the screen and the touch panel 140 is tapped (step S202, Yes), the tablet terminal 100 sets the target area 142 using the line-of-sight position as the center of gravity (step S203). The tablet terminal 100 analyzes the DOM information in the HTML in the target area 142, and specifies the number and coordinates of the links in the target area 142 (step S204).

  If there is no link (step S205, No), the tablet terminal 100 proceeds to step S201. When there is a link (step S205, Yes), the tablet terminal 100 determines the arrangement position of the selection button (step S206).

  The tablet terminal 100 generates correspondence information 152b by associating the image of the selection button with the link coordinates (step S207). The tablet terminal 100 displays the menu image 31 in the default area 143 (step S208).

  The tablet terminal 100 determines whether a touch operation has been received within a predetermined time (step S209). If the tablet terminal 100 does not accept the touch operation within the predetermined time (step S209, No), the tablet terminal 100 proceeds to step S212.

  When the tablet terminal 100 receives the touch operation within the predetermined time (step S209, Yes), the tablet terminal 100 specifies the link coordinates corresponding to the touched selection button based on the correspondence relationship information 152b (step S210). .

  The tablet terminal 100 performs an operation (step S211). In step S211, for example, the tablet terminal 100 displays the link destination of the link coordinates on the touch panel 140.

  The tablet terminal 100 deletes the menu image (step S212), initializes the correspondence information 152b (step S213), and proceeds to step S201.

  FIG. 16 is a flowchart when a moving image is operated. The process illustrated in FIG. 16 is executed when the image data is acquired from the camera 110, for example. As shown in FIG. 16, the tablet terminal 100 detects the line-of-sight position (step S301), and determines whether the line-of-sight position is within the screen and the touch panel 140 is tapped (step S302). If the line-of-sight position is not within the screen or if the touch panel 140 is not tapped (No at Step S302), the tablet terminal 100 proceeds to Step S301.

  On the other hand, when the line-of-sight position is within the screen and the touch panel 140 is tapped (Yes in step S302), the tablet terminal 100 sets the target area 142 using the line-of-sight position as the center of gravity (step S303). The tablet terminal 100 analyzes the DOM information in the HTML in the target area 142 and specifies the moving image in the target area 142 (step S304).

  The tablet terminal 100 determines the arrangement position of the menu image 32 (step S305), and acquires correspondence information 152c corresponding to the moving image (step S306). The tablet terminal 100 displays the menu image 32 in the default area 143 (step S307).

  The tablet terminal 100 determines whether a touch operation has been received within a predetermined time (step S308). If the tablet terminal 100 does not accept the touch operation within the predetermined time (step S308, No), the tablet terminal 100 proceeds to step S311.

  When the tablet terminal 100 receives a touch operation within a predetermined time (step S308, Yes), the tablet terminal 100 determines an operation on the moving image based on the touched button and the correspondence information 152c (step S309).

  The tablet terminal 100 performs an operation (step S310). In step S310, for example, when the touch operation is performed on the reproduction button 32a, the tablet terminal 100 reproduces the moving image in the target area 142. When a touch operation is performed on the stop button 32b, the tablet terminal 100 stops the moving image in the target area 142. When a touch operation is performed on the sound button 32c, the tablet terminal 100 switches between silence and sound of the moving image in the target area 142.

  The tablet terminal 100 deletes the menu image 32 (step S311), initializes the correspondence information 152c (step S312), and proceeds to step S301.

  Next, effects of the tablet terminal 100 according to the first embodiment will be described. The tablet terminal 100 identifies the target area 142 that is operated on the touch panel 140 and determines the content of the touch operation in the target area 142. Then, the tablet terminal 100 generates a menu image including options corresponding to the content of the touch operation, displays the menu image in the default area 143, and accepts the touch operation. Since the predetermined area 143 is an area in which the thumb of the hand holding the tablet terminal 100 can reach, the touch panel 140 can be comfortably operated even when the user operates the tablet with one hand.

  By the way, it is conceivable that the image of the target area 142 is displayed as it is in the default area 143 to allow the user to perform a touch operation, but in that case, the touch operation may not be performed comfortably. For example, when an operation target image or hypertext is small, it is difficult to perform a touch operation. In addition, even when the image is too large with respect to the size of the target area 142, it is similarly difficult to perform a touch operation. On the other hand, since the tablet terminal 100 according to the first embodiment determines the content of the touch operation in the target area 142 and displays the menu image in the default area 143, the comfort is not affected by the size of the image or characters. Operability can be realized.

  In addition, the tablet terminal 100 detects the line-of-sight position on the touch panel 140, specifies the target area 142 including the line-of-sight position, and determines the content of the touch operation in the target area 142. For this reason, it is possible to comfortably operate the region of interest of the user even with one hand.

  Next, the tablet terminal according to the second embodiment will be described. Unlike the tablet terminal 100 according to the first embodiment, the tablet terminal 200 according to the second embodiment is a tablet terminal specialized for enlarging an image or characters in a target area.

  FIG. 17 is a diagram illustrating the configuration of the tablet terminal according to the second embodiment. As illustrated in FIG. 17, the tablet terminal 200 includes a camera 210, a light source 220, a communication unit 230, a touch panel 240, a storage unit 250, and a control unit 260.

  The description regarding the camera 210, the light source 220, the communication unit 230, and the touch panel 240 is the same as the description regarding the camera 110, the light source 120, the communication unit 130, and the touch panel 140 described in the first embodiment, and thus the description thereof is omitted.

  The storage unit 250 is a storage device that stores button image information 251. For example, the storage unit 250 corresponds to a semiconductor memory device such as a RAM, a ROM, or a flash memory, or a storage device such as a hard disk or an optical disk.

  The button image information 251 includes image information of an enlarge button for enlarging the image.

  The control unit 260 includes a line-of-sight detection unit 261, a touch determination unit 262, and a display control unit 263. For example, the control unit 260 corresponds to an integrated device such as an ASIC or FPGA. Moreover, the control part 260 respond | corresponds to electronic circuits, such as CPU and MPU, for example.

  The line-of-sight detection unit 261 is a processing unit that detects the line-of-sight position on the touch panel 240. The line-of-sight detection unit 261 outputs information on the line-of-sight position to the display control unit 263. The process in which the line-of-sight detection unit 261 detects the line-of-sight position is the same as that of the line-of-sight detection unit 161 shown in the first embodiment, and a description thereof will be omitted.

  The touch determination unit 262 is a processing unit that determines whether a touch operation has been performed on the touch panel 240. The touch determination unit 262 outputs a determination result as to whether or not a touch operation has been performed to the display control unit 263.

  The display control unit 263 is a processing unit that enlarges and displays an image on the screen around the line-of-sight position. Note that the display control unit 263 executes the following process using the information indicating that the touch operation has been performed from the touch determination unit 262 as a trigger.

  The display control unit 263 acquires the image of the enlarge button from the button image information 251 and displays the image of the enlarge button in the default area 143. When receiving a touch operation on the enlarge button, the display control unit 263 enlarges the image on the touch panel 240 with the line-of-sight position as the center. For example, the display control unit 263 may determine the image enlargement rate according to the number of touch operations on the enlargement button within a predetermined time, and enlarge the image according to the determined image enlargement rate. For example, the display control unit 263 determines the enlargement ratio based on Expression (1). In Expression (1), n is the number of touch operations within a predetermined time.

  Image magnification (%) = 100 + 10 × n (1)

  Note that the display control unit 263 deletes the enlargement button if the touch operation on the enlargement button is not received within a predetermined time after the enlargement button is displayed.

  Next, a screen image of the tablet terminal 200 according to the second embodiment will be described. FIG. 18 is a diagram illustrating a screen image of the tablet terminal according to the second embodiment. As shown in FIG. 18, the tablet terminal 200 displays an image on the screen (step S40), and detects the line-of-sight position 10a (step S41).

  The tablet terminal 200 displays the enlargement button 33 in the default area 143 (step S42). When the tablet terminal 200 receives a touch operation on the enlargement button 33 (step S43), the tablet terminal 200 enlarges the image around the line-of-sight position 10a (step S44).

  Next, a processing procedure of the tablet terminal 200 according to the second embodiment will be described. FIG. 19 is a flowchart illustrating the processing procedure of the tablet terminal according to the second embodiment. The process illustrated in FIG. 19 is executed, for example, when the image data is acquired from the camera 210. As shown in FIG. 19, the tablet terminal 200 detects the line-of-sight position (step S401), and determines whether the line-of-sight position is within the screen and the touch panel 240 is tapped (step S402). When the line-of-sight position is not within the screen or when the touch panel 240 is not tapped (No at Step S402), the tablet terminal 200 proceeds to Step S401.

  On the other hand, when the line-of-sight position is within the screen and the touch panel 240 is tapped (step S402, Yes), the tablet terminal 200 sets the line-of-sight position as the center of enlargement (step S403). The tablet terminal 200 displays an enlarge button in the default area 143 (step S404).

  The tablet terminal 200 determines whether or not there is a touch operation on the enlargement button within a predetermined time (step S405). If there is no touch operation on the enlarge button within the predetermined time (step S405, No), the tablet terminal 200 deletes the enlarge button (step S406), and proceeds to step S401.

  If there is a touch operation on the enlarge button within a predetermined time (step S405, Yes), the tablet terminal 200 enlarges the image on the screen (step S407), and proceeds to step S406.

  Next, effects of the tablet terminal 200 according to the second embodiment will be described. When the tablet terminal 200 displays an enlarge button in the default area 143 and receives a touch operation on the enlarge button, the tablet terminal 200 enlarges the image around the line-of-sight position. Since the predetermined area 143 is an area in which the thumb of the hand holding the tablet terminal 200 can reach, even when the user operates the tablet terminal 200 with one hand, the image area desired by the user can be comfortably enlarged.

  In the first and second embodiments, the processing of the tablet terminals 100 and 200 has been described. However, the content of the processing described above is an example. In the third embodiment, other processes of the tablet terminal will be described. In the third embodiment, for convenience of explanation, processing will be described using the tablet terminal 100 shown in FIG.

  Another example of the menu image for selecting the link destination will be described. When the number of link destinations included in the target area 142 is large, the number of selection buttons included in the menu image increases, and when all the buttons are displayed in the default area 143, the size of the selection buttons is small. It becomes difficult for the user to touch the selection button. For this reason, when the number of link destinations of the target area 142 is equal to or larger than the predetermined number, the tablet terminal 100 displays a part of the selection button on the menu image and receives the selection button switching instruction. You may make it display by switching.

  20 and 21 are diagrams illustrating examples of other menu images. As shown in FIG. 20, some selection buttons are displayed on the menu image 34. When the user selects the remaining selection buttons, the user touches the arrow 34a. For example, when the display control unit 165 receives a swipe in the downward direction of the arrow 34a, the display control unit 165 displays another selection button. Further, the display control unit 165 may display a menu image 35 as shown in FIG. The display control part 165 displays another button, when a touch operation is received with respect to the switching button 35a, 35b. Further, the display control unit 165 may display the selection button in a pull-down format. By executing such processing by the display control unit 165, even if there are a plurality of link destinations in the target area 142, the user can comfortably select the operation button.

  Next, processing in which the tablet terminal 100 corrects the line-of-sight position will be described. For example, in the first embodiment, as shown in step S102 in FIG. 14, step S202 in FIG. 15, and step S302 in FIG. 16, when the line-of-sight position is within the screen and the touch panel 140 is tapped, the line-of-sight position is set as the center position. As shown in FIG. However, it is conceivable that the angle of the tablet terminal 140 is shifted due to the tap operation, and the line-of-sight position is shifted.

  For this reason, for example, the area specifying unit 162 acquires information on whether or not the user has performed a touch operation from the touch determination unit 164, and uses the line-of-sight position a predetermined time before the touch operation is performed. May be specified. Alternatively, the area specifying unit 162 may calculate the average of the line-of-sight positions from a predetermined time before performing the touch operation to immediately before performing the touch operation, and may set the target area 142 using the calculated average position as the center of gravity. By executing such processing by the area specifying unit 162, it is possible to correct the shift of the line-of-sight position due to the touch operation.

  Next, other processes of the display control unit 165 will be described. In the first embodiment described above, the display control unit 165 has shown an example in which the menu images 30, 31, and 32 are separately displayed in the predetermined area 143, but the present invention is not limited to this. The display control unit 165 may display the menu images 30, 31, and 32 in a composite manner according to the content of the touch operation in the target area 142. For example, when there are a plurality of link destinations and moving images in the target area 142, the display control unit 165 displays the menu images 31 and 32 in the default area 143.

  When there are a plurality of link destinations, it may be difficult to understand the correspondence between the selection button and the link destination simply by displaying the menu image 31. For this reason, the display control unit 165 may visually associate the selection item of the target area 142 with the selection button of the menu image. For example, the display control unit 165 may display the corresponding selection buttons and selection items in the same color, or may be displayed by connecting them with lines or the like. Alternatively, the display control unit 165 may display the selection buttons of the menu image 31 side by side in accordance with the arrangement order of the link destinations of the target area 142. When the display control unit 165 executes such processing, the relationship between the selection button of the menu image and the link destination of the target area 142 is clarified, and the user can select which selection button to perform a touch operation. It will be easier to see if the link destination can be selected.

  Further, the menu image 32 for operating the moving picture shown in FIG. 6 is an example. Depending on the type of moving image in the target area 142, there is one that is automatically played back. For this reason, the display control unit 165 may arrange a pause button instead of the playback button 32a for a video that is automatically played back. Alternatively, an operation button for jumping the reproduction time of the moving image to a desired time may be displayed on the menu image 32.

  By the way, the various processes described in the above embodiments can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. In the following, an example of a computer that executes an operation reception program having the same function as that of the above-described embodiment will be described with reference to FIG.

  FIG. 22 is a schematic diagram illustrating an example of a computer that executes an operation reception program according to the embodiment. As illustrated in FIG. 22, the computer 300 includes an operation unit 301, a speaker 302, a camera 303, a display 304, and a communication unit 305. Further, the computer 300 includes a CPU 306, a ROM 307, a RAM 308, and a hard disk device 309. These units 301 to 309 are connected via a bus 310.

  As shown in FIG. 22, the hard disk device 309 stores an area specifying program 309a, an operation determination program 309b, and a display control program 309c.

  Then, the CPU 306 reads the area specifying program 309a, the operation determination program 309b, and the display control program 309c and expands them in the RAM 308. The area specifying program 309a functions as an area specifying process 308a. The operation determination program 309b functions as an operation determination process 308b. The display control program 309c functions as a display control process 308c.

  For example, the area specifying process 308a performs processing corresponding to the line-of-sight detection unit 161, the area specifying unit 162, and the like illustrated in FIG. The operation determination process 308b performs processing corresponding to the operation determination unit 163 and the like illustrated in FIG. The display control process 308c performs processing corresponding to the touch determination unit 164, the display control unit 165, and the like.

  Note that the programs 309a to 309c are not necessarily stored in the hard disk device 309 or the like from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk inserted into the computer 300, so-called FD, CD-ROM, DVD disk, magneto-optical disk, or IC card. Then, the computer 300 may acquire and execute each program from these portable physical media. Each program is stored in another computer or server device connected to the computer 300 via a public line, the Internet, a LAN, a WAN, etc., and the computer 300 acquires and executes each program from these. It may be.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Appendix 1) An area specifying unit for specifying an area where a touch operation is performed on the touch panel;
An operation determination unit for determining the content of the touch operation in the region specified by the region specifying unit;
A tablet terminal comprising: a display control unit that generates an option corresponding to the content of the touch operation determined by the operation determination unit and displays the generated option in a predetermined area of the touch panel.

(Additional remark 2) It further has a gaze detection part which detects a gaze position on the touch panel,
The tablet terminal according to appendix 1, wherein the area specifying unit specifies an area including the line-of-sight position.

(Additional remark 3) It further has the touch determination part which determines whether touch operation was performed on the said touch panel,
The tablet terminal according to appendix 2, wherein the area specifying unit corrects the position of the area including the line-of-sight position when a touch operation is performed on the touch panel.

(Additional remark 4) When the number of the choices corresponding to the content of touch operation is more than predetermined number, the said display control part displays some choices in a predetermined area | region among several options, It is characterized by the above-mentioned. The tablet terminal according to appendix 1, 2 or 3.

(Additional remark 5) The said display control part displays the touch operation target in the area | region specified by the said area | region specific part, and the said option visually in the predetermined area | region of the said touch panel. The tablet terminal as described in any one of 1-4.

(Appendix 6) An operation reception method executed by a computer,
Identify the area where the touch operation is performed on the touch panel,
Determine the content of the touch operation in the identified area,
An operation receiving method for generating each option corresponding to the content of the determined touch operation and displaying the generated option in a predetermined area of the touch panel.

(Additional remark 7) The process which detects the eyes | visual_axis position on the said touch panel is further performed,
The operation receiving method according to appendix 6, wherein the process of specifying the area specifies an area including the line-of-sight position.

(Additional remark 8) The process which determines whether touch operation was performed on the said touch panel is further performed,
The operation receiving method according to appendix 7, wherein the process of specifying the area corrects the position of the area including the line-of-sight position when a touch operation is performed on the touch panel.

(Additional remark 9) The said process to display displays some choices in a predetermined area | region among several choices, when the number of the choices corresponding to the content of touch operation is more than predetermined number, It is characterized by the above-mentioned. The operation accepting method according to appendix 6, 7 or 8.

(Supplementary note 10) Any one of Supplementary notes 6 to 9, wherein the display processing includes displaying the touch operation target in the specified area and the option in a predetermined area of the touch panel in a visually correlated manner. The method of accepting operations according to any one of the above.

(Supplementary note 11)
Identify the area where the touch operation is performed on the touch panel,
Determine the content of the touch operation in the identified area,
An operation reception program for generating an option corresponding to the content of the determined touch operation and displaying each of the generated option in a predetermined area of the touch panel.

(Additional remark 12) The process which detects the eyes | visual_axis position on the said touch panel is further performed,
The operation reception program according to appendix 11, wherein the process of specifying the area specifies an area including the line-of-sight position.

(Additional remark 13) The process which determines whether the touch operation was performed on the said touch panel is further performed,
The operation receiving program according to appendix 12, wherein the process of specifying the area corrects the position of the area including the line-of-sight position when a touch operation is performed on the touch panel.

(Supplementary note 14) When the number of options corresponding to the content of the touch operation is greater than or equal to a predetermined number, the display process displays some of the options in a predetermined area. The operation reception program according to appendix 11, 12 or 13.

(Supplementary note 15) Any one of Supplementary notes 11 to 14, wherein the display process includes displaying the touch operation target in the specified area and the option in a predetermined area of the touch panel in a visually associated manner. The operation reception program according to any one of the above.

(Supplementary Note 16) Memory,
A processor connected to the memory,
The processor is
Identify the area where the touch operation is performed on the touch panel,
Determine the content of the touch operation in the identified area,
A tablet terminal that generates an option corresponding to the content of the determined touch operation and displays the generated option in a predetermined area of the touch panel.

(Additional remark 17) The area | region where touch operation is performed on a touchscreen is specified,
Determine the content of the touch operation in the identified area,
Generating an option corresponding to the content of the determined touch operation, and displaying the generated option in a predetermined area of the touch panel;
A computer-readable recording medium that stores an operation reception program for causing a computer to execute each process.

100, 200 Tablet terminal 110, 210 Camera 120, 220 Light source 130, 230 Communication unit 140, 240 Touch panel 150, 250 Storage unit 160, 260 Control unit

Claims (9)

A line-of-sight detection unit for detecting a line-of-sight position on the touch panel;
A touch determination unit that determines whether a touch operation has been performed on the touch panel;
When a touch operation is performed on the touch panel, the position of the region including the line-of-sight position is corrected to the position of the region including the line-of-sight position before the touch operation is performed, and the region where the position is corrected is An area specifying unit that specifies an area where a touch operation is performed;
An operation determination unit for determining the content of the touch operation in the region specified by the region specifying unit;
A tablet terminal comprising: a display control unit that generates an option corresponding to the content of the touch operation determined by the operation determination unit and displays the generated option in a predetermined area of the touch panel.   2. The display control unit according to claim 1, wherein, when the number of options corresponding to the content of the touch operation is equal to or greater than a predetermined number, a part of the plurality of options is displayed in a predetermined area. The tablet device described.   3. The display control unit according to claim 1, wherein the touch operation target in the region specified by the region specifying unit and the option are visually associated with each other and displayed in a predetermined region of the touch panel. Tablet device according to. An operation reception method executed by a computer,
Detect line-of-sight position on touch panel,
Determining whether a touch operation has been performed on the touch panel;
When a touch operation is performed on the touch panel, the position of the area including the line-of-sight position is corrected to the position of the area including the line-of-sight position a predetermined time before the touch operation is performed, and the area whose position is corrected is touched. Identify the area where the operation will be performed,
Determine the content of the touch operation in the identified area,
An operation receiving method for generating each option corresponding to the content of the determined touch operation and displaying the generated option in a predetermined area of the touch panel. On the computer,
Detect line-of-sight position on touch panel,
Determining whether a touch operation has been performed on the touch panel;
When a touch operation is performed on the touch panel, the position of the area including the line-of-sight position is corrected to the position of the area including the line-of-sight position a predetermined time before the touch operation is performed, and the area whose position is corrected is touched. Identify the area where the operation will be performed,
Determine the content of the touch operation in the identified area,
An operation reception program for generating an option corresponding to the content of the determined touch operation and displaying each of the generated option in a predetermined area of the touch panel. A touch panel;
The selection target is displayed on the touch panel, and a menu having a plurality of options is displayed in an area located in the lower left corner of the touch panel, and the plurality of options are operations similar to the operations by selecting the selection target. And a control unit that performs menu display control that may include options that can be specified .   The tablet terminal according to claim 6, wherein a shape of the tablet terminal is a substantially rectangular shape, and the lower left corner is a lower left corner of a long side or a short side. A touch panel;
In the state where the user has the lower left corner of the tablet terminal, the selection target displayed on the touch panel is selected in an area located in the lower left corner of the touch panel that is included in the range where the user's thumb can reach A control unit capable of controlling to further display a menu having a plurality of options including options capable of instructing the same operation as the operation in the case where the selection target is visible;
A tablet terminal characterized by comprising:   The tablet terminal according to claim 8, wherein the tablet terminal has a substantially rectangular shape, and the lower left corner is a lower left corner of a long side or a short side.

Images