JP2020074081A - Operation method - Google Patents

Abstract

To provide an input terminal device that does not react to an invalid motion of finger movement when an input operation is performed by detecting a finger position in a non-contact manner and that correctly executes processing according to the intended input operation.SOLUTION: An input terminal device 1 includes a position detection unit 106, a display unit 109a, and an operation processing control unit. The position detection unit 106 detects a position of a position input object operated by a user in a non-contact manner. The display unit 109a displays a cursor based on the position detected by the position detection unit. The operation processing control unit causes the corresponding operation processing to be executed based on the position detected by the position detection unit. The input terminal device 1 has an operation mode that does not execute any operation processing other than cursor movement according to the position of the position input object and an operation mode that executes drawing for character input according to the operation position of a finger. The operation processing control unit switches a plurality of operation modes when the user performs a specific operation via the position input object, and switches the display of the cursor according to the operation mode.SELECTED DRAWING: Figure 1A

Description

  The present invention relates to a small-sized input terminal device and an operation input method that are used by being worn on a wrist or the like.

  2. Description of the Related Art In recent years, mobile information terminals such as smartphones, which perform not only phone calls but also various information searches, email transmission / reception, schedule management, and the like, have rapidly spread. However, these mobile information terminals are often stored in a bag or a pocket when moving, and it is necessary to take out the mobile information terminal from the bag or pocket one by one in order to confirm the received mail. For this reason, a smaller input terminal device such as a wrist watch connected to these portable information terminals by wireless communication has been developed.

  A mobile information terminal such as a smartphone can display information and input with a finger on a touch panel, but a small input terminal device has a small touch panel, which makes it difficult to perform detailed operation input by touching the finger. .. Therefore, a contactless input method using the back of the user's hand as an input surface has been proposed.

  For example, in the wristband type input device described in Patent Document 1, a light emitting unit and a light receiving unit are provided in the main body, and characters can be input by detecting the position of the back of the hand or the fingertip on the palm in a non-contact manner. It is composed.

JP, 2014-106765, A

  In the conventional character input method using a touch panel or the like, when the user's finger is in contact with the panel surface, the character is drawn according to the movement of the finger, and when the user's finger is away from the panel surface, the character is drawn. You can enter the valid period and invalid period of the input separately so that there is no input. With this, when inputting a character consisting of multiple "images" such as line segments and dots, there will be a valid "image" part and an invalid part that moves between "images". By distinguishing, the desired character can be input correctly.

  On the other hand, in the non-contact input method of Patent Document 1 or the like, a plurality of grid points on the back of the hand through which the fingertip has passed are detected, and the input character is recognized from the dot matrix information. However, regarding the passing position of the fingertip, no distinction is made between an effective portion for the "image" forming the character and an invalid portion for moving between the "image" and the "image". That is, there is a problem in that an input character is likely to be erroneously recognized by detecting an invalid part that is not related to the “image” as a part of the character. In other words, the start point and end point of character drawing are unclear. Even if you predict and exclude this invalid part in advance, if the stroke order is different at the time of input, or if you enter a character containing a dakuten or a character with a large number of strokes, the trajectory of the invalid part is not constant, Incorrect input seems inevitable.

  An object of the present invention is to provide an input terminal device that does not react to an invalid motion of a finger when an input operation is performed by detecting the position of a finger in a non-contact manner, and correctly executes a process according to an intended input operation. Is to provide.

  The present invention, in an input terminal device for inputting a user operation via a position input object, includes a position detection unit that detects the position of the position input object operated by the user in a non-contact manner, and the position detection unit. A display unit that displays a cursor based on the detected position, and an operation processing control unit that executes corresponding operation processing based on the position detected by the position detection unit are provided. There is a plurality of operation modes regarding execution, and one of the operation modes includes a state in which no operation processing other than the movement of the cursor according to the position of the position input object is not executed. The unit switches the plurality of operation modes when the user performs a specific operation via the position input object.

  Further, the present invention provides an operation input method for inputting a user operation via a position input object, a position detecting step of detecting a position of the position input object operated by the user in a non-contact manner, and the position detecting step. The display step of displaying a cursor on the display unit based on the position detected in step 1, and the operation processing step of executing the corresponding operation processing based on the position detected in the position detecting step, wherein the operation processing step is the operation There is a plurality of operation modes related to execution of processing, and one of the operation modes includes a state in which no operation processing other than the movement of the cursor according to the position of the position input object is not executed. The processing step switches the plurality of operation modes when the user performs a specific operation via the position input object.

  According to the present invention, when a finger position is detected in a non-contact manner and an input operation is performed, the input terminal device does not react to an invalid movement of the finger movement and correctly executes a process according to an intended input operation. Can be provided.

The block diagram which shows the structure of an input terminal device (Example 1). The figure which shows an example of the software configuration of an input terminal device. The figure which shows the external appearance of an input terminal device. The figure which shows the detection principle of a finger operation mode. The figure which shows the example of a display of a display part in a handwritten character input process. The flowchart which shows the operation switching of the whole input terminal device. The figure which shows the operation mode and the example of the cursor display corresponding to it. The figure which shows the example of the screen display in a handwritten character input process. FIG. 8 is a diagram showing a change with time of an operation mode and a drawing position in FIG. 7. The flowchart which shows the handwritten character input process by finger operation mode. 10 is a flowchart showing operation mode determination processing S605 in FIG. 9. The figure which shows the structure of the display part for key character input processing (Example 2). The figure which shows the example of a key character input process by finger operation mode. The flowchart which shows the key character input process by finger operation mode. The flowchart which shows the determination processing S1105 of the operation mode in FIG. FIG. 11 is a diagram showing an example of an operation mode and cursor display used in mouse operation input processing (Example 3). The figure which shows the example of the screen display in mouse operation input processing. The flowchart which shows the mouse operation input process by finger operation mode. 18 is a flowchart showing an operation mode determination process S1504 in FIG. The figure which shows the detection principle of 3D finger operation mode (Example 4). The figure which shows the example of a screen display in 3D finger operation mode. The flowchart which shows the mouse operation input process by 3D finger operation mode. The figure which shows the example of the screen display in key character input processing. FIG. 13 is a diagram showing an example of a screen display when a transition region is provided between layers (Example 5). 9 is a flowchart showing mouse operation input processing when a transition area is provided between layers.

  Embodiments of the present invention will be described below with reference to the drawings.

  In the first embodiment, the configuration of the input terminal device and the case of detecting a finger position and inputting handwritten characters will be described.

  FIG. 1A is a block diagram showing the configuration of the input terminal device 1 according to the first embodiment, and the internal configuration (hardware configuration) will be described. The input terminal device 1 is used by connecting to a mobile information terminal 2 such as a smartphone. The input terminal device 1 of the present embodiment has a finger operation mode (non-contact detection method) using light reflection and a touch operation mode (contact detection method) as operation input functions.

  The main control unit 101 is configured by a CPU (Central Processing Unit) or the like, and controls the entire input terminal device 1 according to various operation programs and data stored in a ROM (Read Only Memory) 102 or a RAM (Random Access Memory) 103. Control. The system bus 100 is a data communication path for transmitting and receiving data between the control unit 101 and each unit in the input terminal device 1.

  The ROM 102 is a memory in which various programs for controlling the input terminal device 1 are stored, and a rewritable ROM such as an EEPROM (Electrically Erasable Programmable ROM) or a flash ROM is used. The RAM 103 is used as a temporary storage area when the program stored in the ROM 102 is executed or as temporary storage of finger position information detected by the finger position detection sensor 106. The storage unit 104 stores information such as operation setting values of the input terminal device 1, and uses a nonvolatile rewritable device such as a flash ROM or SSD (Solid State Drive). The ROM 102 and the RAM 103 may be integrated with the main controller 101. Further, the ROM 102 may be configured to use a partial storage area in the storage unit 104 instead of having an independent configuration.

  The acceleration sensor 105 is a sensor that detects the acceleration of the input terminal device 1. The finger position detection sensor 106 is a sensor that detects the position of a position input object such as a user's finger or a pen by reflected light in a non-contact manner. The short-range wireless communication unit 107 connects to the portable information terminal 2 by wireless communication and transmits the information input to the input terminal device 1. As a communication method, for example, Bluetooth (registered trademark) or WiFi (registered trademark) is used.

  The display control unit 108 controls to display the input characters and various setting images of the input terminal device 1 on the display unit 109a. The touch panel 109 includes a display unit 109a and a touch operation input unit 109b. The display unit 109a is a display device such as a liquid crystal panel, and displays characters and images under the control of the display control unit 108. The touch operation input unit 109b is arranged on the display surface of the display unit 109a.

  The touch operation input unit 109b is, for example, a touch pad type input device such as a capacitance type, and detects a touch operation by a finger, a touch pen, or the like as an operation input. Specifically, for example, a gesture called swipe that touches the touch panel with the finger and then moves the finger in a specific direction while touching, touches the touch panel with the finger, and then moves the finger at a speed equal to or higher than a predetermined value Detects a gesture called a flick that releases a touch, a gesture called a tap that quickly releases after touching the touch panel with a finger, and the like. As a result, a touch-type operation input to the input terminal device 1 becomes possible.

  The clock unit 110 measures the elapsed time from the date / time set by the user using an RTC (Real Time Clock) circuit, and outputs date / time information. The main control unit 101 performs processing such as a timer by reading the date / time information output from the clock unit 110. The operation input unit 111 is an instruction input unit that inputs an operation instruction to the input terminal device 1, and has buttons for turning on / off the power and setting an operation mode.

  As described above, the input terminal device 1 according to the present embodiment includes a finger position detection sensor 106 that detects a position of a user's finger in a non-contact manner and inputs a character or the like, and a touch operation input by touching the finger with the touch panel 109. The operation input unit 109b and the display unit 109a for displaying characters and images are provided. This enables two operation input functions of the finger operation mode and the touch operation mode.

  FIG. 1B is a diagram showing an example of the software configuration of the input terminal device 1, and shows the software configuration of the ROM 102, the RAM 103, and the storage unit 104.

  The main control unit 101 configures a basic operation processing control unit 1021, a finger position information acquisition unit 1022, and a touch position information acquisition unit 1023 by executing a program stored in the ROM 102. The RAM 103 also includes a temporary storage area 1031 that temporarily holds data as needed when executing a program stored in the ROM 102. The storage unit 104 stores setting information 1041 for executing the programs stored in the ROM 102. It should be noted that, for simplicity of description, it is described below that the main control unit 101 executes a program stored in the ROM 102 to control each operation block.

  The basic operation processing control unit 1021 controls various settings of the input terminal device 1 and the overall operation. The finger position information acquisition unit 1022 acquires the position information of the position input target object such as the user's finger or pen based on the output of the finger position detection sensor 106. The touch position information acquisition unit 1023 acquires position information when a finger, a touch pen, or the like touches the touch panel 109 based on the output of the touch operation input unit 109b.

  The finger operation processing control unit 1024 determines what operation has been input based on the position information of the position input object acquired from the finger position information acquisition unit 1022, and controls the processing corresponding to the determination result. To do. The touch operation processing control unit 1024 determines what operation has been input based on the information of the position touched on the touch panel 109 acquired from the touch position information acquisition unit 1023, and controls the processing corresponding to the determination result. To do.

  FIG. 2 is a diagram showing an appearance of the input terminal device 1. It is a wristwatch type input terminal device, and is used by being worn on the user's wrist. A touch panel 109 (display unit 109a) is arranged on the front side of the device. Two sensors 106a and 106b are arranged as finger position detection sensors 106 on the back side of the user's hand on the side of the device. These sensors are, for example, light emitting / light receiving elements. Further, an operation button is arranged as the operation input unit 111 on the side of the device on the side of the user's arm.

  FIG. 3 is a diagram showing the detection principle of the finger operation mode, and shows the relationship between the position of the finger and the display position of the display unit. The user's finger (position input object) PS moves, for example, within the position A-B-C-D on the back side of the user's hand. The finger position detection sensors 106a and 106b each include an infrared light emitting element and a light receiving element. The infrared light emitted from the light emitting element is reflected by the finger PS, and the light receiving element detects the reflected light. The level of the detection signal at the light receiving element corresponds to the distances La and Lb from the sensors 106a and 106b to the finger PS.

  The output signal of the finger position detection sensor 106 is sent to the finger position information acquisition unit 1022, and the position F of the finger PS in the position input detection range Pa represented by the positions ABCD is acquired. Further, the positions A-B-C-D of the position input detection range Pa correspond to the positions a-b-c-d of the display range Da of the display unit 109a of the touch panel 109, respectively. Accordingly, the display position f of the cursor or the like in the display range Da is determined in correspondence with the position F of the finger PS in the position input detection range Pa.

  FIG. 4 is a diagram showing a display example of the display unit 109a in the handwritten character input process. Area 501 displays the result of recognizing the input handwritten character, and area 502 displays the handwritten character input by the finger. As the button 503 selected by the user for character input processing, a recognition processing instruction button 503a for selecting processing for recognizing handwritten characters, a character type switching processing instruction button 503b for switching the type of characters to be input such as hiragana and alphabet, and handwriting input A deletion processing instruction button 503c for deleting the display of characters, a character conversion processing instruction button 503d for selecting processing for converting the character string of the recognition result displayed in the area 501 into Chinese characters, and an end processing instruction for ending the handwritten character input processing. A button 503e and the like are arranged.

  FIG. 5 is a flowchart showing the operation switching of the entire input terminal device 1. Here, by pressing a button on the operation input unit 111, power on / off and operation input mode (finger operation mode / touch operation mode) are switched.

  In S301, the output of the operation input unit 111 is fetched, and in S302, it is determined whether or not the operation input unit 111 is pressed. If the operation input unit 111 is pressed (Yes), the process branches to S306. In S306, the time during which the operation input unit 111 is pressed is determined, and the process proceeds to power-on / off switching or operation input mode switching processing.

  When the operation input unit 111 is pressed for the predetermined time Tp or more in the determination process S306 (Yes), the process branches to S313, and the process of switching on / off is performed depending on whether the power is on. If the power is on in S313 (Yes), the flow branches to S314 to turn off the power and return to S301. When the power is turned off, in order to reduce the power consumption, the power supply to the parts other than the main control unit 101, the clock unit 110, the operation input unit 111 and the like is stopped. When the power is off in S313 (No), the process branches to S315, the power is turned on, and the process returns to S301. When the power is turned on, the power is supplied to each unit that constitutes the input terminal device 1.

  If the operation input unit 111 has not been pressed for the predetermined time Tp or more in the determination processing S306 (No), the process branches to S307 and it is determined whether or not the power is on. When the power is off (No), the process returns to S301. If the power is on (Yes), the process branches to S308.

  In S308, operation input mode switching processing is performed. That is, when the current operation input mode is set to the finger operation mode (Yes), the touch operation mode is switched and set in S309, and the cursor is not displayed on the display unit 109a of the touch panel 109 in S310. Set to. When the current operation input mode is not set to the finger operation mode (No) (that is, when it is set to the touch operation mode), the finger operation mode is switched and set in S311 and the touch panel 109 is set in S312. The display is set to display the cursor on the display unit 109a.

  If the operation input unit 111 is not pressed in the determination process S302 (No), the process proceeds to S303 and the process is performed according to the set operation input mode. In the case of the finger operation mode (Yes), the process proceeds to S304, it is determined what operation is input based on the finger position information acquired from the finger position information acquisition unit 1022, and the process corresponding to the determination result is performed. Run. If the operation mode is not the finger operation mode (No) (that is, if the operation mode is the touch operation mode), the process proceeds to S305, and what operation is input based on the touched position information acquired from the touch position information acquisition unit 1023. Is determined and the process corresponding to the determination result is executed. After the process in the finger operation execution process S304 or the touch operation execution process S305 is executed, the process returns to S301.

  The handwritten character input process in the finger operation mode will be specifically described below. In the present embodiment, a plurality of operation states (operation modes) are given to a finger operation so that handwritten character input can be correctly and easily performed.

  FIG. 6 is a diagram showing an example of operation modes and cursor display corresponding thereto. The operation mode M0 is a state in which drawing is not performed for character input other than cursor movement even when a finger is operated (drawing stopped). The operation mode M1 is a state in which when a finger is operated, drawing for character input is executed according to the position of the finger. Further, the cursor display (for example, white / black display) is switched according to the operation mode so that the user can easily identify the current operation mode. The operation mode is switched from M0 to M1 or vice versa by performing a specific operation of holding a finger being operated at a position on the operation surface for a predetermined time t0 or more, as described later. Can be switched from M1 to M0.

  As a result, drawing is possible only when the operation mode is M1. Therefore, when inputting a character composed of a plurality of "images" such as line segments and dots, the operation mode is set to M1 for the effective "image" portion, and the "image" and "image" are combined. By setting the operation mode to M0 for an invalid portion that moves between spaces, desired characters can be easily and correctly input.

  FIG. 7 is a diagram showing an example of a screen display in the handwritten character input process. Here, the case of inputting the hiragana "a" is taken as an example. The character input procedure is shown by (S1) to (S9), and the operation mode (M0 / M1) at that time is additionally described.

  In the following description, the cursor is expressed as "holding" at a certain position, but even if the position of the cursor moves (blur) within a predetermined distance r0, this is tolerated and regarded as a "holding" state. And This r0 is due to a movement (camera shake or the like) not intended by the user, and is set to a value of, for example, several mm. The time t0 for determining the "holding" state is, for example, a value of about several seconds. These thresholds may be set according to the user and the usage environment.

(S1): In the state where the operation mode is M0 (drawing stopped), the finger PS is moved within the position input detection range Pa to move the cursor to the position 7a of the character input display area 502.
(S2): Hold the cursor at the position 7a for a predetermined time t0 or more. As a result, the operation mode is switched to M1 (drawing execution), and the cursor display is also changed.

(S3): When the display of the cursor is changed, the finger is moved to move the cursor to the position 7b of the character input display area 502. Thereby, the line from the position 7a to the position 7b is drawn.
(S4): Hold the cursor at the position 7b for a predetermined time t0 or more. As a result, the operation mode is switched to M0 (drawing stopped) and the cursor display is changed.

(S5): The finger is moved when the display of the cursor is changed, and the cursor is moved to the position 7c of the character input display area 502.
(S6): Hold at position 7c for a predetermined time t0 or more. As a result, the operation mode is switched to M1 and the cursor display is changed.

  Thereafter, the drawing operation is similarly continued. That is, when drawing is performed, the operation mode is M1 and the cursor is moved, and when drawing is not performed, the operation mode is M0 and the cursor is moved. Further, the operation mode is switched by holding the cursor at the moved position for a predetermined time t0 or more, thereby inputting handwritten characters.

(S7): When the drawing of the hiragana "A" is completed, it is held at the position 7d for a predetermined time t0 or more. As a result, the operation mode is switched to M0 and the cursor display is changed.
(S8): The cursor is moved to the recognition processing instruction button 503a.
(S9): The recognition processing is executed by holding the cursor on the recognition processing instruction button 503a for a predetermined time t0 or more, and the recognition result is displayed in the display area 501.

  FIG. 8 is a diagram showing the change over time in the operation mode and the drawing position in FIG. (A) shows the position of the position input object (finger of the user) PS, (b) shows the operation mode, and (c) shows the drawing position of the display unit. The symbols (S1) to (S6) and symbols (7a) to (7c) on the horizontal axis correspond to the symbols in FIG. In this way, drawing is performed only in the period in which the operation mode is M1, and the operation mode is switched by holding the position input object PS at the same position for time t0 or more.

  FIG. 9 is a flowchart showing a handwritten character input process in the finger operation mode. The contents of each process will be described below in order.

  In step S601, the finger position information acquisition unit 1022 acquires the position F of the finger PS, and in step S602, the acquired position F of the finger PS is converted into the position f in the display range Da as described with reference to FIG. Next, in S603, it is determined whether or not the distance r between the position f and the previously acquired position fd stored in the temporary storage area of the RAM 103 is less than or equal to a predetermined distance r0.

  If the distance r is equal to or greater than the predetermined distance r0 in the determination processing S603 (No), the process branches to S617. In S617, it is determined whether the position f is within the display range of the character input display area 502. If the position f is within the range of the character input display area 502 (Yes), the process branches to S618. If the position f is outside the range of the character input display area 502 (No), the process branches to S620. In S620, the operation mode is set to M0, and the process proceeds to S615.

  In S618, the set operation mode is determined. When the operation mode is M0, the drawing process is not performed and the process proceeds to S615. When the operation mode is M1, the process branches to S619, the drawing process from the position fd to the position f in the display range Da is performed, and the process proceeds to S615.

  On the other hand, if the distance r is equal to or less than the predetermined distance r0 in the determination processing S603 (Yes), the process branches to S604. In S604, it is determined whether or not the position f is within the display range of the predetermined area 502 for inputting characters in the display range Da or within the display range of the processing instruction button 503. When the position f is within the display range of the character input display area 502 or the processing instruction button 503 (Yes), the process proceeds to S605. If the position f is outside the display range of the character input display area 502 or the processing instruction button 503 (No), the process proceeds to S615.

  In S605, the operation mode is determined by the timer. That is, the operation time is switched by comparing the holding time with the predetermined time t0. Details of the determination processing in S605 will be described with reference to FIG. In S606, branch processing is performed according to the determined operation mode. When the operation mode is M0, the process proceeds to S615, and when the operation mode is M1, the process proceeds to S607. In S607, it is determined whether the position f is within the display range of the processing instruction button 503. If the position f is within the display range of the processing instruction button 503 (Yes), the process proceeds to S608, and if the position f is outside the display range of the processing instruction button 503 (No), the process proceeds to S615.

  In S608, branch processing is performed depending on which processing instruction button display range from 503a to 503e the position f is. When the position f is within the display range of the recognition processing instruction button 503a, the processing proceeds to S609 and the recognition processing is executed. When the position f is within the display range of the character type switching process instruction button 503b, the process proceeds to S610 and the character type switching process is executed. If the position f is within the display range of the erasing process instruction button 503c, the process proceeds to S611 and the erasing process is executed. If the position f is within the display range of the conversion process instruction button 503d, the process proceeds to S612 and the conversion process is executed. After each processing is executed, the process proceeds to S613 to reset / stop the timer, and the operation mode is set to M0 in S614. If the position f is within the display range of the termination processing instruction button 503e, the handwritten character input processing is terminated.

  In S615, for example, a cursor as shown in FIG. 6 is displayed at the position f of the display range Da according to the set operation mode. Next, in S616, the current position f is replaced as the previous position fd, stored in the temporary storage area of the RAM 103, and the process returns to S601.

  FIG. 10 is a flowchart showing the operation mode determination processing S605 in FIG.

  In S701, it is determined whether the timer has been started. If the timer is not activated (No), the process proceeds to S702, activates the timer, and ends the process. When the timer is activated (Yes), the process proceeds to S703.

  In S703, it is determined whether the timer has exceeded the predetermined time t0. When it does not exceed the predetermined time t0 (No), the processing is ended. If it exceeds the predetermined time t0 (Yes), the process proceeds to S704.

  In S704, branch processing is performed according to the operation mode. When the operation mode is M0, the process proceeds to S705, and the operation mode is switched to M1 and set. When the operation mode is M1, the process proceeds to S706, and the operation mode is switched to M0 and set. After that, the process proceeds to S707, the timer is reset / stopped, and the process ends.

  As described above, in the first embodiment, when the position on the display screen of the position input object such as the finger of the user is held within the predetermined distance r0 for a predetermined time t0 or more to switch the operation mode and perform drawing. Is set to M1 and the cursor is moved, and when drawing is not performed, the operation mode is set to M0 and the cursor is moved. This makes it possible to distinguish movement between strokes as a non-drawing section even when inputting characters with multiple strokes, and to easily enter the intended characters correctly even if the stroke order is different. Is possible. It is also possible to input more complicated characters such as kanji.

  Further, the processing corresponding to the processing instruction button can be performed by moving the cursor to the area where the processing instruction button is displayed and holding the position within a predetermined distance r0 from the position where the cursor is moved for a predetermined time t0 or more. It is possible to perform the same operation as when the cursor is moved to the position where the processing instruction button is displayed by the input means such as a mouse and then clicked.

  In the present embodiment, the processing instruction buttons include the recognition processing instruction button 503a, the character type switching processing instruction button 503b, the deletion processing instruction button 503c, the character conversion processing instruction button 503d, and the end processing instruction button 503e. However, the present invention is not limited to this, and other processing instruction buttons may be provided.

  In the present embodiment, the operation input mode (touch operation mode) by the touch panel 109 and the operation input mode (finger operation mode) by the finger position detection sensor 106 are switched by the operation input unit 111. You may make it switch. For example, when the back of a hand is tapped with a finger, vibration is detected by the acceleration sensor 105 and touch is not detected by the touch panel 109. Therefore, it is determined from the detection result of the touch panel 109 and the detection result of the acceleration sensor 105 that the back of the hand is tapped. However, the operation input mode may be switched when the back of the hand is tapped with a finger.

  Further, in this embodiment, as a specific operation for switching the operation mode, an operation of holding the position of the position input object such as the user's finger on the display screen within a predetermined distance r0 for a predetermined time t0 or more is adopted. However, it is not limited to this. As another specific operation, for example, the operation mode may be switched by moving the cursor by moving the position input object and tapping the back of the hand at the moved position.

  In the second embodiment, a case will be described in which input is performed by selecting a key displayed on the screen like a keyboard. In a small portable information terminal, the keys displayed on the display unit 109a are also small, and it is difficult to input by touching the touch panel 109 with a finger. Therefore, key characters are input in the finger operation mode.

  FIG. 11 is a diagram showing the configuration of the display unit 109a for the key character input process. An area 1001 is a portion for displaying a character input by a key character, and a character input key 1002 is a key for selecting a character to be input. A key 1003 is for instructing various kinds of processing, and is a character type switching processing instruction key 1003a for switching the type of characters to be input such as hiragana or alphabet, and a deletion processing instruction key for deleting the input character displayed in the area 1001. Reference numeral 1003b includes an end processing instruction key 1003c for ending the key character input processing.

  FIG. 12 is a diagram showing an example of key character input processing in the finger operation mode, and here, the case of inputting the alphabetical character “Y” is taken as an example. The input procedure will be described in (S1) to (S4).

(S1): In the state where the operation mode is M0, by moving the finger PS within the position input detection range Pa, the cursor is moved to the position 12a where the character input key "WXYZ" is displayed.
(S2): The cursor is held at the position 12a for a predetermined time t0 or more (also in this case, movement within a predetermined distance r0 is allowed. The same applies hereinafter). As a result, the operation mode is switched and set to M1, and the display for selecting the characters “W”, “X”, “Y”, and “Z” is performed.

(S3): When the character is selected and displayed, the finger is moved to move the cursor to the position 12b where the character "Y" is displayed.
(S4): Hold the cursor at the position 12b for a predetermined time t0 or more. As a result, the character “Y” is determined as the input character, and the character “Y” is displayed in the input character display area 1001. When the character input is completed, the operation mode is switched to M0.

  FIG. 13 is a flowchart showing a key character input process in the finger operation mode.

  In S1101, the position F of the finger PS is acquired by the finger position information acquisition unit 1022, and in S1102, the acquired position F of the finger PS is converted into the position f in the display range Da. Next, in S1103, it is determined whether or not the distance r between the position f and the previously acquired position fd is less than or equal to a predetermined distance r0.

  In the determination processing S1103, if the distance r is equal to or greater than the predetermined distance r0 (No), the process branches to S1114. If the distance r is less than or equal to the predetermined distance r0 (Yes), the process branches to S1104. In S1104, it is determined whether the position f is within the display area of the character input key 1002 or the processing instruction key 1003. In the determination process S1104, if the position f is within the display area of the character input key 1002 or the processing instruction key 1003 (Yes), the process branches to S1105, and if the position f is outside the display area of the character input key 1002 or the processing instruction key 1003 (No. ), The process branches to S1114.

  In S1105, the operation mode is determined by the timer. That is, the operation time is switched by comparing the holding time with the predetermined time t0. Details of the determination processing in S1105 will be described with reference to FIG. In S1106, branch processing is performed according to the determined operation mode. When the operation mode is M0, the process proceeds to S1114, and when the operation mode is M1, the process proceeds to S1107. In step S1107, it is determined whether the character selection display is on.

  In the determination process S1107, if the character selection display is on (Yes), the process proceeds to S1116, and if the character selection display is off (No), the process proceeds to S1108. In S1116, it is determined whether the position f is within the character selection display area. If the position f is within the character selection area (Yes), the process proceeds to S1117, and if the position f is outside the character selection display region (No), the process proceeds to S1114. In S1117, the character displayed at the position f is determined as the input character and displayed in the input character display area 1001. In S1118, the character selection display is turned off, and the process proceeds to S1112.

  In S1108, branch processing is performed depending on whether the position f is within the display area of the character input key 1002 or the processing instruction key 1003. If the position f is within the display area of the character input key 1002, the flow advances to S1109 to turn on the character selection display, and the character corresponding to the key displayed at the position f is selected and displayed. If the position f is within the display area of the character type switching process instruction key 1003a, the process proceeds to S1110 to execute the character type switching process. If the position f is within the display area of the deletion processing instruction key 1003b, the process proceeds to S1111 to execute the deletion processing. After each processing is executed, the process proceeds to S1112, the timer is reset / stopped, and the operation mode is set to M0 in S1113. When the position f is within the display area of the termination processing instruction key 1003c, the key character input processing is terminated.

  In S1114, for example, a cursor as shown in FIG. 6 is displayed at the position f of the display range Da according to the set operation mode. Next, in S1115, the current position f is replaced with the previous position fd, stored in the temporary storage area of the RAM 103, and the process returns to S1101.

  FIG. 14 is a flowchart showing the operation mode determination processing S1105 in FIG.

  In S1201, it is determined whether the timer has been started. If the timer is not activated (No), the process proceeds to S1202, activates the timer, and ends the process. If the timer is activated (Yes), the process proceeds to S1203.

  In S1203, it is determined whether the timer has exceeded the predetermined time t0. When it does not exceed the predetermined time t0 (No), the processing is ended. When it exceeds the predetermined time t0 (Yes), the process proceeds to S1204.

  In S1204, the operation mode is switched to M1 and set. In this case, in S1113 of FIG. 13, the operation mode after execution of the key character input is switched to M0, so switching from M1 to M0 is unnecessary. After that, the process proceeds to S1205, the timer is reset / stopped, and the process ends.

  As described above, in the second embodiment, the cursor is moved to the position where the character input key or the processing instruction key is displayed, and the position within a predetermined distance r0 from the position of the moved cursor is held for a predetermined time t0 or more for input. It is possible to easily select a character to be executed or instruct a process to be executed. As a result, the same operation as that when the cursor is moved to the position where the character input key or the processing instruction key is displayed by the input means such as a mouse and then clicked can be performed.

  In the present embodiment, the processing instruction keys are composed of the character type switching processing instruction key 1003a, the deletion processing instruction key 1003b, and the end processing instruction key 1003c, but the present invention is not limited to this, and other processing instructions can be used. Corresponding keys may be provided. As for the character input keys, a keyboard in which one letter of the alphabet or hiragana corresponds to one key, or a keyboard corresponding to the input of numbers may be displayed and input.

  In the third embodiment, operations such as clicking with a mouse, double clicking, and drag and drop (hereinafter referred to as mouse operation input processing) are realized in the finger operation mode. In order to deal with this, the number of types of operation modes is increased in this embodiment.

  FIG. 15 is a diagram showing an example of the operation mode and cursor display used in the mouse operation input process. Here, six types of operation modes M0 to M5 are used. The operation mode M0 is a state in which no input operation other than cursor movement is performed even if a finger is operated. In the operation modes M1 to M5, mouse operations such as a click process (M1), a double click process (M2), a drag process start (M3), a drag process end (M4), and a drop process (M5) are performed. is there. The cursor display (shape and / or color display) is switched according to each operation mode. It should be noted that the switching of the operation mode is performed by comparing the length of time for which the finger being operated is held at a certain position on the operation surface with three threshold values t0, t1, t2, as described later.

  FIG. 16 is a diagram showing an example of a screen display in the mouse operation input process. On the display unit 109a, an icon group 1401 corresponding to the applications A to D is displayed as a window screen (home screen). The user can select a desired application by clicking the icon, execute the application by double-clicking it, and move the icon display position by dragging and dropping. A series of processing flow will be described in (S1) to (S7).

(S1): In the state where the operation mode is M0, by moving the finger within the position input detection range, the cursor is moved to the position 14a where the icon [D] is displayed.
(S2): Hold the cursor at the position 14a for a predetermined time (range from t0 to t1). As a result, the operation mode is switched to M1 (click processing), and the icon “D” is selected as the mouse operation target. Further, the display of the cursor changes to the display corresponding to M1 in FIG. After that, by arbitrarily moving the finger until the predetermined time t1 elapses (in this case, the cursor is moved to a position away from the position 14a by a predetermined distance r0 or more), the selected icon “D” is displayed. Click processing is executed.

  (S3): When the cursor is held at the position 14a for a predetermined time t1 or longer, the operation mode is switched to M2 (double-click processing), and the cursor is changed to the display corresponding to M2 shown in FIG. After that, by arbitrarily moving the finger until the predetermined time t2 elapses (the cursor is moved from the position 14a by a predetermined distance r0 or more), the double-click process is executed on the selected icon “D”. ..

(S4): When the cursor is held at the position 14a for a predetermined time t2 or longer, the operation mode is switched to M3 (drag processing start), and the cursor is changed to the display corresponding to M3 shown in FIG.
(S5): Then, the operation mode is switched to M4 (drag processing end) by moving the finger to move the cursor from the position 14a to the position 14b which is a predetermined distance r0 or more. As a result, the cursor changes to a display corresponding to M4 shown in FIG. Further, the icon “D” selected as the mouse operation target is moved and displayed at the position 14b where the cursor is moved, and the drag processing for moving the position of the selected icon is executed.

(S6): The finger is further moved to move the cursor and the selected icon [D] to the position 14c, and the cursor is held at the position 14c for a predetermined time t0 or more. As a result, the operation mode is switched to M5 (drop processing), and the drop processing is executed at the position 14c.
(S7): By the drop processing, the display position of the selected icon "D" is set to 14c. After that, the operation mode is switched to M0, and the cursor becomes a display corresponding to M0.

  FIG. 17 is a flowchart showing a mouse operation input process in the finger operation mode.

  In S1501, the finger position information acquisition unit 1022 acquires the position F of the finger PS, and in S1502, the acquired position F of the finger PS is converted into a position f in the display range Da. Next, in S1503, it is determined whether or not the distance r between the position f and the previously acquired position fd is less than or equal to a predetermined distance r0.

  In the determination process S1503, if the distance r is equal to or less than the predetermined distance r0 (Yes), the process branches to S1504, and in S1504, the operation mode is determined by the timer. That is, the holding time is compared with the predetermined times t0, t1, and t2 to switch the operation mode. Details of the determination processing in S1504 will be described with reference to FIG. After that, in S1505, it is determined whether the operation mode determined in S1504 is M5. If the operation mode is M5 (Yes), the process branches to S1508, and if the operation mode is not M5 (No), the process branches to S1506. In S1508, a drop process of setting the display position of the icon selected by the drag process to the position f is executed, and the process proceeds to S1509.

  In step S1506, it is determined whether the position f is within the icon display area. If the position f is within the icon display area (Yes), the process branches to S1507, and if it is outside the icon display region (No), the process branches to S1509. In S1507, the icon whose position f is within the display area is selected as a mouse operation target, and the process proceeds to S1518. In S1509, the operation mode is set to M0, and the process proceeds to S1510. In S1510, the timer is reset / stopped, and the process proceeds to S1518.

On the other hand, in the determination processing S1503, if the distance r is equal to or greater than the predetermined distance r0 (No), the process branches to S1511. In S1511, the following branch processing is performed according to the operation mode.
When the operation mode is M0, the process is not performed and the process proceeds to S1518.
When the operation mode is M1, the process proceeds to S1515, the icon selected as the mouse operation target in S1507 is clicked, and the process proceeds to S1516.
When the operation mode is M2, the process proceeds to S1514, the icon selected as the mouse operation target in S1507 is double-clicked, and the process proceeds to S1516.
If the operation mode is M3, the operation proceeds to S1513, the operation mode is set to M4, and the operation proceeds to S1517.
If the operation mode is M4, the process advances to step S1512, and the icon selected as the mouse operation target in step S1507 is displayed at the position f. As a result, a drag process for moving the position of the selected icon is performed, and then the process proceeds to S1518.

  In S1516, the operation mode is set to M0, in S1517 the timer is reset / stopped, and the process proceeds to S1518. In S1518, the cursor as shown in FIG. 15 is displayed at the position f of the display range Da according to the current operation mode. Next, in S1519, the current position f is replaced with the previous position fd, stored in the temporary storage area of the RAM 103, and the process returns to S1501.

  FIG. 18 is a flowchart showing the operation mode determination processing S1504 in FIG.

  In S1601, it is determined whether the timer has been started. If the timer is not activated (No), the process proceeds to S1602, activates the timer, and ends the process. If the timer has been started (Yes), the process proceeds to S1603.

  In S1603, it is determined whether the operation mode is M4. When the operation mode is M4 (Yes), the process proceeds to S1604. In S1604, it is determined whether the timer has exceeded the predetermined time t0, and if it has not exceeded the predetermined time t0 (No), the process ends. If it exceeds the predetermined time t0 (Yes), the process proceeds to S1605, the operation mode is set to M5, and the process ends. In the determination processing S1603, if the operation mode is not M4 (No), the process proceeds to S1606.

In S1606, the following branch processing is performed according to the elapsed time of the timer. Note that the following time thresholds have a relationship of t0 <t1 <t2.
When the timer does not exceed the predetermined time t0 (timer ≦ t0), the processing is ended.
When the timer is the time between the predetermined times t0 and t1 (t0 <timer ≦ t1), the process proceeds to S1609, the operation mode is set to M1, and the process ends.
When the timer is the time between the predetermined times t1 and t2 (t1 <timer ≦ t2), the process proceeds to S1608, the operation mode is set to M2, and the process ends.
If the timer has exceeded the predetermined time t2 (timer> t2), the process advances to step S1607 to set the operation mode to M3, and the process ends.

  As described above, in the third embodiment, the operation mode is switched to M0 to M5 according to the length of time for holding the position within the predetermined distance r0 from the position where the cursor is moved. Then, after the operation mode is switched, by moving the cursor to a position separated by a predetermined distance r0 or more like a flick operation in a touch operation of a smartphone, a mouse click, a double click, a drag and drop, etc. The operation can be realized in the finger operation input mode. Further, it is possible to confirm that the operation mode has been switched by the change in the display of the cursor, and it becomes possible to easily perform the mouse operation intended by the user.

  In the fourth embodiment, a case will be described in which the position of the user's finger is three-dimensionally detected and an input operation is performed (hereinafter, referred to as a 3D finger operation mode).

  FIG. 19 is a diagram showing the detection principle of the 3D finger operation mode, and shows the relationship between the position of the finger and the display position of the display unit. The position of the finger PS is detected not only in the direction parallel to the display unit 109a of the input terminal device 1 (XY axis direction) but also in the vertical direction (height Z axis direction), and the input operation is performed. The figure shows the relationship between the detected position F of the finger (position input object) PS and the display position f of the display unit 109a.

  In order to detect the finger position three-dimensionally, three finger position detection sensors 106a, 106b, 106c are provided. Each of the sensors 106a to 106c includes, for example, an infrared light emitting element and a light receiving element, and each of the sensors 106a to 106c outputs a signal of a level corresponding to the distances La, Lb, and Lc to the finger PS. In the finger position information acquisition unit 1022, the position input detection range (space) surrounded by the positions A-B-C-D-A'-B'-C'-D 'based on the output signals from the sensors 106a to 106c. The position F of the finger PS at Pa is acquired. The position f in the display range Da surrounded by the positions abcd of the display unit 109a is determined corresponding to the acquired position F in the XY axis directions, and, for example, a cursor or the like is displayed. On the other hand, depending on the position of the acquired position F in the Z-axis direction, a plurality of display screens are switched as described later.

  FIG. 20 is a diagram showing an example of a screen display in the 3D finger operation mode. Here, a window screen (home screen) in which icons are arranged is displayed as an operation screen. The position in the height Z direction with respect to the display unit 109a is divided into a plurality of layers L0, L1 and L2 with Z0 to Z3 as boundaries. The window screen is switched and displayed depending on which layer the finger PS is located. Display states in each layer are shown by (S1) to (S3).

(S1): When the position F0 of the finger PS is in the range of the height directions Z0 to Z1 (layer L0), the icon group 1901 corresponding to the applications A to D is displayed on the window screen.
(S2): When the position F1 of the finger PS is in the range of the height directions Z1 to Z2 (layer L1), the icon group 1902 corresponding to the applications E to H is displayed on the window screen.
(S3): When the position F2 of the finger PS is in the range of the height directions Z2 to Z3 (layer L2), the icon group 1903 corresponding to the applications I to L is displayed on the window screen.

  After that, the mouse operation input processing described in the third embodiment is executed. That is, an application is selected by clicking the displayed icon, the application is executed by double-clicking the icon, and the display position of the icon is moved by dragging and dropping the icon.

  As described above, according to the 3D finger operation mode, the screen display for scanning can be switched for each of a plurality of layers, so that the mouse operation input process can be easily performed for a large number of applications (icons).

  FIG. 21 is a flowchart showing mouse operation input processing in the 3D finger operation mode. The same processes as those in the third embodiment (FIG. 17) are designated by the same reference numerals, and repeated description will be omitted.

  In S1501, the three-dimensional position F of the finger PS is acquired by the finger position information acquisition unit 1022, and in S1502, the acquired position in the XY direction is converted into the position f in the display range Da as described with reference to FIG. .. Next, in S1520, it is determined which layer the position of the acquired position F in the Z direction is within. In S1521, it is determined whether the determined layer LN is the same as the previously determined layer LNd stored in the temporary storage area of the RAM 103. As a result of the determination, when the determined layer LN is the same as the previously determined layer LNd (Yes), the process proceeds to S1503. Since the processing after S1503 is the same as that in FIG. 17, the description thereof will be omitted. As a result of the determination, if the determined layer LN is different from the previously determined layer LNd (No), the process proceeds to S1522.

  In S1522, it is determined whether the operation mode is M3 or M4. When the operation mode is M3 or M4 (Yes), the process proceeds to S1523, and when the operation mode is not M3 or M4 (No), the process proceeds to S1525. In S1525, the operation mode is set to M0, and the process proceeds to S1526. In S1526, the timer is reset / stopped, and the flow proceeds to S1523.

  In S1523, the discriminated layer LN is replaced as the discriminating layer LNd of the previous time, stored in the temporary storage area of the RAM 103, and the process proceeds to S1524. In S1524, the screen of the display unit 109a is updated to display the window screen (home screen) corresponding to the determined layer LN, and the process proceeds to S1511. The process after S1511 is a mouse operation input process according to each operation mode M0 to M4, but it is the same as that of FIG. 17 and its explanation is omitted.

  As described above, by detecting the three-dimensional position of the user's finger and performing an operation input, the user's finger is divided into a plurality of layers according to the position in the height Z direction, and a window screen for operation corresponding to each layer ( Home screen) can be displayed. It is also possible to perform mouse operations such as click, double click, drag and drop on the icons displayed in the windows of each layer.

  In the above example, the icons corresponding to the different applications for each of the plurality of layers are displayed on the window screen, but the present invention is not limited to this, and it is possible to display different operation surfaces for each layer.

  For example, FIG. 22 is a diagram showing an example of a screen display in the key character input process. In the key character input process, a keyboard of a character type different for each layer is displayed on the display unit.

(S1): When the position F0 of the finger PS is within the range of the layer L0, the keyboard for inputting alphabets and symbols is displayed.
(S2): When the position F1 is within the range of the layer L1, the keyboard for inputting hiragana is displayed.
(S3): If the position F2 is within the range of the layer L2, the keyboard for inputting numbers is displayed.
Accordingly, the user can easily change the character type to be input by changing the position of the finger PS in the height Z direction.

  As described above, according to the fourth embodiment, the screen display and the operation content can be switched for each of a plurality of layers in the 3D finger operation mode, so that various kinds of operations can be easily realized. The number of layers may be appropriately set by the user according to the type of operation used.

  The fifth embodiment will explain a configuration in which a transition region is provided between layers in order to smoothly switch between a plurality of layers in the 3D finger operation mode.

  FIG. 23 is a diagram showing an example of a screen display when a transition area is provided between layers. The position of the finger PS in the height Z direction is divided into a plurality of layers L0 to L2, and layer transition regions T01 and T12 are provided at the boundary of each layer. Similar to the fourth embodiment, different operation window screens (here, icon groups) are displayed corresponding to the respective layers L0 to L2, but in the layer transition areas T01 and T12, corresponding to any adjacent layers. Display the window screen. Hereinafter, a change in screen display will be described along the movement of the finger PS in the height direction.

  First, a case where the position of the finger PS is moved from F0 to F4 (that is, layers L0 to L2) will be described in (S1) to (S5).

(S1): The position F0 of the finger PS is within the range of the layer L0, and the icon group 2101 corresponding to the applications A to D is displayed on the window screen.
(S2): When the position F1 of the finger PS moves to the layer transition area T01 between the layers L0 and L1, the window screen (icon group 2101) displayed on the immediately previous layer L0 is continuously displayed. At that time, the screen background is changed to the display 109T indicating the transition area.

(S3): The position F2 of the finger PS is within the range of the layer L1, and the icon groups 2102 corresponding to the applications E to H are switched and displayed on the window screen.
(S4): When the position F3 of the finger PS moves to the layer transition area T12 between the layers L1 and L2, the window screen (icon group 2102) displayed on the immediately previous layer L1 is continuously displayed. At that time, the screen background is changed to the display 109T indicating the transition area.

  (S5): The position F4 of the finger PS is within the range of the layer L2, and the icon groups 2103 corresponding to the applications I to L are switched and displayed on the window screen.

  On the other hand, a case where the position of the finger PS is moved from F4 to F0 (that is, layers L2 to L0) will be described in (S5 ') to (S1').

(S5 ′): The position F4 of the finger PS is within the range of the layer L2, and the icon group 2103 corresponding to the applications I to L is displayed on the window screen.
(S4 ′): When the position F3 of the finger PS moves to the layer transition area T12 between the layers L1 and L2, the window screen (icon group 2103) displayed on the immediately previous layer L2 is continuously displayed. At that time, the screen background is changed to the display 109T indicating the transition area.

(S3 ′): The position F2 of the finger PS is within the range of the layer L1, and the icon groups 2102 corresponding to the applications E to H are switched and displayed on the window screen.
(S2 ′): When the position F1 of the finger PS moves to the layer transition area T01 between the layers L0 and L1, the window screen (icon group 2102) displayed on the immediately previous layer L1 is continuously displayed. At that time, the screen background is changed to the display 109T indicating the transition area.

  (S1 '): The position F0 of the finger PS is within the range of the layer L0, and the icon groups 2101 corresponding to the applications A to D are switched and displayed on the window screen.

  As described above, when the finger PS is located within the range of each layer, the window screen corresponding to each layer is displayed, and when the finger PS moves to the layer transition area, the window screen of the layer located immediately before is displayed. Is continuously displayed and the background is changed and displayed.

  In this way, the transition area is provided between the layers, and when the finger PS moves to the transition area, the background of the window screen is changed to notify the user of the movement to the transition area. At that time, since the window screen displayed immediately before is continued as it is, it does not hinder the operation. This can prevent the user from unintentionally moving to another layer. That is, when the background of the window screen changes, the finger PS may be moved in the direction of the original layer.

  FIG. 24 is a flowchart showing mouse operation input processing when a transition area is provided between layers. The same processes as those in the third embodiment (FIG. 17) and the fourth embodiment (FIG. 21) are designated by the same reference numerals, and repeated description will be omitted.

  In S1501, the three-dimensional position F of the finger PS is acquired, and in S1502, the position of the position F in the XY directions is converted into the position f in the display range Da. Next, in step S1520, it is determined which layer the position F in the Z direction is within or the layer transition region. In S1530, branch processing is performed according to the layer discrimination result.

  In the determination processing S1530, if the determined layer LN is within the layer transition area (LN = between layers), the process branches to S1531. In S1531, the background of the window display screen is set to the display 109T corresponding to the layer transition area, and the process proceeds to S1503.

  In the determination processing S1530, if the determined layer LN is the same as the previously determined layer LNd (LN = LNd), the process proceeds to S1503. Since the processing after S1503 is the same as that in FIG. 17, the description thereof will be omitted.

  In the determination processing S1530, when the determined layer LN is different from the previously determined layer LNd (LN ≠ LNd), the process proceeds to S1522. Since the processing after S1522 is the same as that in FIG. 21, the description thereof is omitted.

  As described above, according to the fifth embodiment, not only the three-dimensional position of the finger PS is detected and the windows corresponding to the plurality of layers are displayed corresponding to the position in the height direction Z, but also between the layers. A transition area is provided. Then, when the finger PS moves to the transition area, the background of the window screen is changed to notify the user, and it is possible to prevent the user from unintentionally moving to another layer. This allows smooth switching between layers.

  Although the background of the window screen is changed when the finger PS moves to the layer transition area in this embodiment, the present invention is not limited to this. For example, any method that can be recognized by the user may be used, such as changing the icon display, changing the cursor display, or generating sound or vibration. Further, when it is located in the transition area, the direction (up / down) of the original layer may be displayed.

  In the above description of each embodiment, an example in which the finger position detection sensor 106 is configured by a light emitting / light receiving element has been shown, but the present invention is not limited to this. Any device capable of detecting the position of a position input object such as a pen or a pen may be used. Although the wristwatch type input terminal device has been described as an example, a smartphone, a tablet, a personal computer, or the like detects the position of a position input target object such as a user's finger or a pen to perform handwritten character input or mouse operation. You may do it.

  In each of the above embodiments, the position of the position input object such as the user's finger on the display screen is held within a predetermined distance for a predetermined time or more as a condition of a specific operation for switching the operation mode. However, it is not limited to this. For example, the condition may be that the movement amount of the position input object on the display screen within a predetermined time is within a predetermined distance. Alternatively, the condition may be that the moving speed of the position input object on the display screen is less than or equal to a predetermined speed.

  Although the embodiment of the present invention has been described with reference to some examples, it goes without saying that the configuration for realizing the technique of the present invention is not limited to the above-described examples, and various modifications can be considered. .. For example, a part of the configuration of a certain embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of a certain embodiment. All of these belong to the category of the present invention. Also, the numerical values, messages, etc. appearing in the text and in the figures are merely examples, and the use of different values does not impair the effects of the present invention. In addition, the programs described in each processing example may be independent programs, or a plurality of programs may constitute one application program. Further, the order of performing each process may be changed and executed.

  The functions and the like of the present invention described above may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Alternatively, the microprocessor unit or the like may be realized by software by interpreting and executing an operation program that realizes each function or the like. Hardware and software may be used together.

  Further, the control lines and information lines shown in the drawings are those considered necessary for the description, and not all the control lines and information lines on the product are necessarily shown. In practice, it may be considered that almost all configurations are connected to each other.

  1: Input terminal device, 2: Mobile information terminal, 101: Main control unit, 102: ROM, 103: RAM, 104: Storage unit, 105: Acceleration sensor, 106, 106a, 106b, 106c: Finger position detection sensor, 108 : Display control unit, 109: Touch panel, 109a: Display unit, 109b: Touch operation input unit, 110: Timing unit, 111: Operation input unit, 1022: Finger position information acquisition unit, 1023: Touch position information acquisition unit, 1024: Finger operation processing control unit, 1025: touch operation processing control unit, PS: user's finger (position input object).

Claims (9)

An operation method for operating a terminal device via a position input object, comprising:
A position detection step of detecting the position of the position input object in a non-contact manner,
A display step of displaying an icon capable of selecting an object on the display unit of the terminal device based on the position detected in the position detection step;
An operation processing step of causing the terminal device to execute a corresponding operation processing based on the position detected in the position detection step,
In the operation processing step, switching between a plurality of operation modes regarding execution of the operation processing including an operation mode in which any operation processing other than movement of the icon is not executed as an operation according to the position of the position input object, An operation method, which can be executed by an operation of holding the position of the position input object within a predetermined range for a predetermined time or more. The operation method according to claim 1, wherein
In the display step, the operation method is characterized in that at least one of the shape and the color of the icon is changed according to the plurality of operation modes and displayed on the display unit. The operation method according to claim 1, wherein
The plurality of operation modes include a first operation mode in which the icon is moved and drawn on the display unit in accordance with the movement of the position detected in the position detection step, and the position detected in the position detection step. An operation method comprising a second operation mode in which only the icon is moved on the display unit according to the movement. The operation method according to claim 1, wherein
The operating method further comprises a transmitting step of transmitting information from the terminal device to a portable information terminal wirelessly connected to the terminal device based on the operating process in the operating process step. An operation method for operating a terminal device via a position input object, comprising:
A position detection step of detecting the position of the position input object in a non-contact manner,
A display step of displaying an icon capable of selecting an object on the display unit of the terminal device based on the position detected in the position detection step;
An operation processing step of causing the terminal device to execute a corresponding operation processing based on the position detected in the position detection step,
In the operation processing step, switching between a plurality of operation modes regarding execution of the operation processing including an operation mode in which any operation processing other than movement of the icon is not executed as an operation according to the position of the position input object, Executable by performing a specific operation via the position input object,
Furthermore, in the operation processing step, a processing instruction button is displayed on the display unit, and the processing is performed when the position of the icon is held at a position where the processing instruction button is displayed within a predetermined range for a predetermined time or more. An operation method characterized in that a process corresponding to an instruction button is executed. The operating method according to claim 5, wherein
The operating method further comprises a transmitting step of transmitting information from the terminal device to a portable information terminal wirelessly connected to the terminal device based on the operating process in the operating process step. An operation method for operating a terminal device via a position input object, comprising:
A position detection step of detecting the position of the position input object in a non-contact manner,
A display step of displaying an icon capable of selecting an object on the display unit of the terminal device based on the position detected in the position detection step;
An operation processing step of causing the terminal device to execute a corresponding operation processing based on the position detected in the position detection step,
In the operation processing step, switching between a plurality of operation modes regarding execution of the operation processing including an operation mode in which any operation processing other than movement of the icon is not executed as an operation according to the position of the position input object, Executable by performing a specific operation via the position input object,
Further, in the operation processing step, the position in the vertical direction with respect to the display unit is divided into a plurality of layers, and the position of the position input object detected in the position detecting step falls within the range of any one of the plurality of layers. An operation method characterized by switching an operation screen displayed on the display unit depending on whether there is an operation screen. The operation method according to claim 7, wherein
In the operation processing step, a transition area is further set at the boundary of the plurality of layers, and when the position of the position input object moves into the transition area, the operation previously displayed on the display unit. The operating method is characterized in that the background of the screen is changed while the screen for the game is continuously displayed. The operation method according to claim 7, wherein
The operating method further comprises a transmitting step of transmitting information from the terminal device to a portable information terminal wirelessly connected to the terminal device based on the operating process in the operating process step.

Images