JP5845585B2 - Information processing device - Google Patents

Description

  The present invention relates to an information processing device, and more particularly to an information processing device including an input device having an operation surface.

  In recent years, information terminal devices (information processing devices) typified by smartphones have become widespread, and can be connected to a network to browse homepages, send and receive e-mails, download applications, etc. It is supposed to be. Such an information terminal device has conventionally been provided with operation buttons, direction keys, or a touch screen (touch type user interface) to which functions are assigned in advance, and inputs information by user operation.

  In this type of information terminal device, the operation content with respect to the operation unit such as the operation button is diversified due to the downsizing and the spread of applications, and a comfortable operability is required. Therefore, various input devices capable of inputting various information in a limited space have been proposed. For example, Patent Document 1 discloses an input device that improves operability by using fluctuations in the output value of a coordinate position signal.

  By the way, in the information terminal device as described above, since an operation unit such as an operation button or a touch-type user interface is arranged in the vicinity of the display screen, the user can easily view desired information while comparing the display screen and the operation source. You can enter. On the other hand, when the operation unit and the display screen are arranged at positions separated from each other, the user cannot easily compare the display screen and the operation source. Therefore, when operating a touch-type user interface, in a general input method that recognizes that an execution instruction is input with a user's touch operation, the execution instruction is input to unintended coordinates, Inconvenience occurred, such as starting an unintended application.

  In such information terminal devices, various technologies have been proposed that allow a user to operate without looking at the operation source. For example, Patent Document 2 discloses an information processing apparatus that improves input operability by allowing a user to recognize a function assigned to an input unit before performing an input operation.

Japanese Patent No. 4465231 JP 2004-355606 A

  However, the technique disclosed in Patent Document 1 or Patent Document 2 has a problem that a drag operation or a flick operation performed on a general touch pad cannot be recognized.

  A general touch-type user interface is arranged on the display screen, and the user can operate the object by moving the operating body to a position to be operated while simultaneously viewing the object displayed on the display screen and the operation by the user. However, when the touch pad is provided separately from the display screen, the user must operate the object displayed on the display screen in a state where the position specified by the touch pad is not clear. In addition, when a general touch pad is operated, a click operation is performed at the moment of touching the touch pad. As described above, when the touch pad and the display screen are provided separately, the user cannot determine the position of the finger operated by the user and is determined to be a click operation at the moment of touching the touch pad. There was a problem of significant reduction.

  The present invention has been made in view of the above technical problems. According to some aspects of the present invention, it is possible to provide an information processing apparatus or the like that realizes various operability without looking at the operation source even when the operation surface and the display screen are provided separately. it can.

(1) In the first aspect of the present invention, the information processing apparatus has a one-to-one correspondence between the coordinate position in the operation region and the coordinate position of the display region by the display device, and is separated from the display region by the display device. has not that operation surface provided Te, an input device for detecting whether the non-pressing state press state in which a predetermined or more pressure is applied to the contact position and the operating surface of the operating body in the operation surface, the A processing device that executes processing corresponding to the operation of the operation body on the operation surface, and the processing device is configured to perform the processing in the display area when the operation body is in contact with the operation surface in the non-press state. position location mark on the display position corresponding to the contact position is displayed in the first display mode, when the operating body shifts to the press state in the operation surface, and the said localization mark the first display form Made and displayed on the second display mode, the is moved from the first contact position has shifted to the press state to the second contact position while the pressing state, the transition said to non pressing state at the contact position of the second At this time, the drag process corresponding to the drag operation determined by the first contact position and the second contact position is distinguished from the flick process corresponding to the flick operation.

In this embodiment, the coordinate position of the operation area is associated on a one-to-one to the coordinate position of the display area by the display device, an input device having an operating surface that provided separately from the display area by the display device, The contact position of the operation body on the operation surface and whether the pressure is applied to the operation surface in a pressed state or a non-pressed state are detected. When the operating body is in contact with the operation surface in the non-pressed state, the position specifying mark is displayed in the first display mode at the display position corresponding to the contact position in the display area. On the other hand, when shifting to the pressed state , the position specifying mark is displayed in a second display mode different from the first display mode, and the contact position when shifting to the pressed state and the contact position when shifting to the non-pressed state The processing corresponding to the operation determined by is performed. By doing so, contact with the operation surface is not handled as a click operation as in the conventional case, and the user who operates can be informed of the position of the operation body before the click operation. Furthermore, since the processing corresponding to the drag operation and the flick operation is separately performed in the pressed state and the non-pressed state and the movement of the contact position, the operation surface and the display area of the display device are separated. However, the user can input a variety of information without looking at the operation source. In addition, the user can input various information while easily discriminating his / her operation without looking at the operation source.

  (2) In the second aspect of the present invention, the information processing apparatus has an operation surface in which the coordinate position in the operation region is associated with the coordinate position of the display region by the display device on a one-to-one basis. An input device that detects a pressed position or a non-pressed state in which a predetermined pressure or more is applied to the contact position of the operating body and the operation surface, and a process corresponding to the operation of the operating body on the operation surface is executed. A processing device, and the processing device displays a position specifying mark at a display position corresponding to the contact position in the display area when the operating body is in contact with the operation surface in the non-pressed state. In the pressed state, a click process corresponding to a click operation is executed at a display position corresponding to the contact position in the display area.

  In this aspect, the input device having the operation surface in which the coordinate position in the operation region is associated with the coordinate position of the display region by the display device on a one-to-one basis with respect to the contact position and operation surface of the operation object on the operation surface It is detected whether it is in a pressed state or a non-pressed state where a pressure higher than a predetermined pressure is applied. When the operating body is in contact with the operation surface in the non-pressed state, a position specifying mark is displayed at a display position corresponding to the contact position in the display area. On the other hand, in the press state, click processing corresponding to the click operation is executed at the display position corresponding to the contact position in the display area. An input device such as a normal pen input digitizer displays an icon even when the pen is away from the operation surface. On the other hand, in this aspect, the position specifying mark is hidden when the operating tool is moved away from the operation surface, but the position specifying mark can be displayed when the operating tool is brought into contact with the operating surface. By doing this, the touch on the operation surface is not treated as a click operation as in the past, and the operating user can be informed of the position of the operating body before the click operation, and corresponds to the contact position in the display area. The confirmation instruction at the displayed position can be performed. In addition, it is possible to accept input of information by distinguishing between the contact of the operating body on the operation surface and the confirmation instruction, and even when the operation surface and the display area of the display device are separated, the user can identify his / her operation source. A variety of information can be input without looking.

  (3) In the information processing apparatus according to the third aspect of the present invention, in the second aspect, after the processing apparatus has maintained the contact position within a predetermined range in the pressed state for a long click determination period or longer. When the state is shifted to the non-press state, a long click process corresponding to a long click operation is executed at a display position corresponding to the contact position in the display area.

  In this aspect, after the contact position is maintained for the long click determination period or longer in the pressed state, the long click process corresponding to the long click operation is performed when the non-pressed state is entered. As a result, in addition to the above effects, it is possible to provide an information processing apparatus capable of inputting a wider variety of information.

  (4) According to a fourth aspect of the present invention, the information processing apparatus has an operation surface in which the coordinate position in the operation region is associated with the coordinate position of the display region by the display device on a one-to-one basis. An input device that detects a pressed position or a non-pressed state in which a predetermined pressure or more is applied to the contact position of the operating body and the operation surface, and a process corresponding to the operation of the operating body on the operation surface is executed. A processing device, and the processing device displays a position specifying mark at a display position corresponding to the contact position in the display area when the operating body is in contact with the operation surface in the non-pressed state. The operating body is moved from the first contact position in the operation surface to the pressed state to the second contact position in the pressed state, and the non-pressed state is moved to the second contact position. When migrated performs flick process corresponding to the drag process or flick operation corresponding to the first contact position and drag operation determined by said second contact position.

  In this aspect, the input device having the operation surface in which the coordinate position in the operation region is associated with the coordinate position of the display region by the display device on a one-to-one basis with respect to the contact position and the operation surface of the operation object on the operation surface. It is detected whether it is in a pressed state or a non-pressed state where a pressure higher than a predetermined pressure is applied. When the operating body is in contact with the operation surface in the non-pressed state, a position specifying mark is displayed at a display position corresponding to the contact position in the display area. On the other hand, processing corresponding to the operation determined by the contact position when shifting to the pressed state and the contact position when shifting to the non-pressed state is performed. By doing so, contact with the operation surface is not handled as a click operation as in the conventional case, and the user who operates can be informed of the position of the operation body before the click operation. Furthermore, since the processing corresponding to the drag operation or the flick operation is performed in the pressed state and the non-pressed state and the movement of the contact position, even when the operation surface and the display area of the display device are separated, the user can , You will be able to input a variety of information without looking at your source.

  (5) In the information processing device according to the fifth aspect of the present invention, in the fourth aspect, the processing device has a movement amount per unit time from the first contact position to the second contact position. The flick process is performed when the value is equal to or greater than a given threshold value, and the drag process is performed when the movement amount per unit time is less than the threshold value.

  According to this aspect, it is possible to input information by distinguishing a drag operation and a flick operation, and it is possible to input a wider variety of information.

  (6) In the information processing device according to the sixth aspect of the present invention, in any one of the second to fifth aspects, the processing device is configured such that when the operation body is in a non-contact state with the operation surface, A process of hiding the position specifying mark in the display area is performed.

  According to this aspect, even when the operation surface and the display area of the display device are provided separately, the information processing device that realizes the same operation as that of a conventional smartphone or the like without looking at the operation source Will be able to provide.

  (7) In the information processing apparatus according to the seventh aspect of the present invention, in any one of the second aspect to the sixth aspect, the processing device is in the display area when in the non-pressed state and in the contact state. The position specifying mark displayed on the display area is different from the position specifying mark displayed in the display area in the pressed state in the contact state.

  According to this aspect, it is possible to provide an information processing apparatus capable of inputting a variety of information while easily discriminating the user's operation without looking at the operation source.

  (8) In the information processing apparatus according to the eighth aspect of the present invention, in any one of the first aspect to the seventh aspect, the processing device is an object or background that overlaps the position specifying mark in the display area. A process of displaying the position specifying mark through the image is performed.

  In this aspect, the user displays the position specifying mark in the display area without looking at the operation source. At this time, since the user can recognize the object or the like originally present at the position where the position specifying mark is displayed, the operability of the user can be further improved.

  (9) In the information processing device according to the ninth aspect of the present invention, in any one of the first to eighth aspects, the input device includes a touch pad having the operation surface and a movable touch pad. The switch is switched when the pressure is applied to the operation surface, and the pressed state or the non-pressed state is detected according to the switching state of the switch.

  According to this aspect, it is possible to realize the simplification and cost reduction of the configuration of the information processing apparatus having the above effects.

  (10) In the information processing device according to the tenth aspect of the present invention, in the ninth aspect, the input device moves the movable position of the touch pad to the switch switching position in the pressed state, It includes one or a plurality of cushion materials that return the movable position of the touch pad to a predetermined position when in the non-pressed state.

  According to this aspect, it is possible to provide an input device that detects whether it is in a pressed state or a non-pressed state with a very simple configuration, and the configuration of the information processing device having the above effects is further simplified. And cost reduction can be realized.

  (11) An information processing apparatus according to an eleventh aspect of the present invention includes the display device according to any one of the first aspect to the tenth aspect, in which a processing result by the processing apparatus is reflected.

  According to this aspect, there is provided an information processing apparatus that includes a display device and realizes various operability without looking at the operation source even when the operation surface and the display area of the display device are provided separately. Will be able to.

  (12) In the information processing apparatus according to the twelfth aspect of the present invention, in the eleventh aspect, the display device is a head mounted display device.

  According to this aspect, even when inputting information while looking at a head-mounted display device that makes it difficult to look at the operation source for the purpose, various operability is realized without the user looking at the operation source. It becomes possible to provide an information processing apparatus.

The block diagram of the example of composition of the information processor concerning one embodiment of the present invention. The figure which shows typically an example of the image which a user uses the information processing apparatus of FIG. FIG. 3A is a functional block diagram illustrating a configuration example of the input device described in this embodiment. FIG. 3B is a functional block diagram illustrating a configuration example of the input device in the case where some of the functions of FIG. FIG. 4A is a schematic view of the operation surface of the input device of FIG. 1 as viewed from above. FIG. 4B is a schematic view of the input device shown in FIG. Explanatory drawing of the operation surface of the input device of FIG. FIG. 6A is a diagram schematically showing a cross-sectional structure along the line AA ′ in FIG. 4B in a state where the user does not touch the operation surface with the finger. FIG. 6B is a diagram schematically showing a cross-sectional structure along the line AA ′ of FIG. 4B in a state where the user applies pressure to the operation surface with a finger. FIG. 7A is a diagram illustrating a display example of a pointer in a non-press state. FIG. 7B is a diagram showing a display example of a pointer in a pressed state. FIG. 8A is an explanatory diagram of a pointing operation before the finger is brought into contact with the operation surface. FIG. 8B is an explanatory diagram of a pointing operation when a finger is brought into contact with the operation surface. FIG. 8C is an explanatory diagram of the pointing operation when the finger is horizontally moved while being in contact with the operation surface. FIG. 8D is an explanatory diagram of a pointing operation when a finger is released from the operation surface. FIG. 9A is an explanatory diagram of the click operation before the finger is brought into contact with the operation surface. FIG. 9B is an explanatory diagram of the click operation when the finger is brought into contact with the operation surface. FIG. 9C is an explanatory diagram of a click operation when pressure is applied to the operation surface with a finger. FIG. 10A is an explanatory diagram of a click operation when the pressure applied to the operation surface with a finger is weakened. FIG. 10B is an explanatory diagram of a click operation when a finger is released from the operation surface. FIG. 11A is an explanatory diagram of a drag operation before bringing a finger into contact with the operation surface. FIG. 11B is an explanatory diagram of a drag operation when a finger is brought into contact with the operation surface. FIG. 11C is an explanatory diagram of a drag operation when pressure is applied to the operation surface with a finger. FIG. 12A is an explanatory diagram of a drag operation when the contact position is moved horizontally while changing the pressure on the operation surface with a finger. FIG. 12B is an explanatory diagram of a drag operation when a finger is released from the operation surface. The functional block diagram of the structural example of the processing apparatus of FIG. The figure which shows the hardware structural example of the processing apparatus of FIG. The flowchart of the process example of a processing apparatus. The flowchart of the process example of a processing apparatus. FIG. 17 is a flowchart of a processing example of the drag processing in FIG. 16. FIG. 17 is a flowchart of a processing example of the flick processing in FIG. 16. The figure which shows typically the other example of the image which a user uses the information processing apparatus of FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The embodiments described below do not unduly limit the contents of the present invention described in the claims. In addition, all of the configurations described below are not necessarily indispensable configuration requirements for solving the problems of the present invention.

FIG. 1 shows a block diagram of a configuration example of an information processing apparatus according to an embodiment of the present invention.
FIG. 2 schematically shows an example of an image in which the user uses the information processing apparatus of FIG. FIG. 2 shows an example in which the display device applied to the information processing apparatus in the present embodiment is a head mounted display (hereinafter referred to as HMD), but the display device according to the present embodiment is an HMD. It is not limited to.

  As shown in FIG. 1, the information processing apparatus 10 according to the present embodiment includes an information input device 20 and a display device 30, and the information input device 20 and the display device 30 are connected via a wired or wireless signal transmission path 22. It is connected. Operation information corresponding to an operation by the user is input to the information input device 20. The display device 30 displays a result reflecting the operation information by displaying an image corresponding to the operation information input to the information input device 20.

  The information input device 20 includes an input device 100 and a processing device 200. The input device 100 has an operation surface. The coordinate position in the operation area on the operation surface is associated with the coordinate position of the display area by the display device 30 on a one-to-one basis. Further, the input device 100 is in a pressed state or a non-pressed state in which a predetermined pressure or more is applied to the contact position of the operation body on the operation surface and the operation surface (no more than a predetermined pressure is applied to the operation surface). Status). Hereinafter, the operation tool will be described as the finger of the user who operates the operation tool. In this case, the input device 100 can include a capacitive touch pad or a resistive touch panel. The processing device 200 executes processing corresponding to the operation of the finger on the operation surface included in the input device 100.

  In such a configuration, the processing device 200 is a so-called pointing operation, a click operation, a long click operation, based on the contact position of the finger on the operation surface and whether or not a predetermined pressure is applied to the operation surface. Determine whether dragging or flicking. The processing device 200 determines these operations and executes processing corresponding to the determined operations. An image corresponding to the processing result of the processing device 200 is displayed on the display device 30.

  As shown in FIG. 2, the user 24 wears a display device 30 realized by the HMD on the head. Since the HMD has a known configuration, a detailed description of the HMD is omitted. The display device 30 and the information input device 20 are connected by, for example, a wired signal transmission path 22. When information is input by the user 24 in the information input device 20, an image corresponding to the input information is displayed on the display screen of the HMD. That is, the operation surface and the display screen are provided separately. Since the user 24 sees the display screen of the HMD, the user 24 cannot operate while looking at the operation surface of the information input device 20.

  However, in the present embodiment, when the finger of the user 24 is in contact with the operation surface of the input device 100, the pointer is displayed at a display position corresponding to the contact position in the display area of the HMD, thereby Can teach the position of. Further, the user 24 can issue a confirmation instruction at the display position corresponding to the contact position in the display area by applying pressure to the operation surface of the input device 100 with a finger. Thereby, in the information input device 20, the input of information is accepted by distinguishing the contact of the finger on the operation surface and the confirmation instruction. As a result, the user 24 can input a variety of information without looking at the operation source of the user 24.

[Information input device]
Hereinafter, the input device 100 and the processing device 200 constituting the information input device 20 of the information processing device 10 in the present embodiment will be described in detail.

[Input device]
3A and 3B are functional block diagrams of a configuration example of the input device 100 in FIG. FIG. 3A illustrates a configuration example of the input device described in this embodiment. FIG. 3B illustrates a configuration example of the input device in the case where some of the functions of FIG. 3A and 3B, parts that are the same as those in FIG. 1 are given the same reference numerals, and descriptions thereof will be omitted as appropriate.

  The input device 100 constituting the information input device 20 includes a position detection unit 110 and a press detection unit 120 as shown in FIG. In addition, the input device 100 may include a position detection unit 110 as illustrated in FIG. 3B, and the function of the press detection unit 120 in FIG. 3A may be provided outside the input device 100. Hereinafter, the input device 100 will be described as having the configuration of FIG.

  The position detection unit 110 detects the contact position of the finger on the operation surface, and outputs position information corresponding to the contact position to the processing device 200. Such a function of the position detection unit 110 is realized by, for example, the above-described capacitive touch pad. In the capacitive touch pad, a contact position is detected by a change in capacitance caused by a finger touching the operation surface. Note that the function of the position detection unit 110 may be realized by, for example, the above-described resistance film type touch panel. In the resistive film type touch panel, the position is detected when the upper electrode and the lower electrode come into contact with each other when the flexible operation surface to which pressure is applied is recessed. Therefore, the state in which the upper electrode and the lower electrode are in contact with each other by the pressure of the finger in the resistive touch panel corresponds to the state in which the finger is in contact with the operation surface in the capacitive touch pad.

  The press detection unit 120 detects whether the pressure is applied to the operation surface of the input device 100 with a finger, and outputs the press information indicating the pressed state or the non-pressed state to the processing device 200. Such a function of the press detection unit 120 is realized by the following configuration, for example.

FIG. 4A shows a schematic view of the operation surface of the input device 100 of FIG. 1 as viewed from above. FIG. 4B shows a schematic view of the input device 100 shown in FIG.
FIG. 5 is an explanatory diagram of an operation surface of the input device 100 of FIG. FIG. 5 shows the correspondence between the operation surface of the input device 100 and the display screen of the display device 30 of FIG.

As shown in FIG. 4A, the input device 100 includes a touch pad 102 having an operation surface AR whose operation region OR is a rectangular shape, and the user touches the finger 26 at an arbitrary position in the operation surface AR. Thus, a corresponding position in the display screen (display area) 32 of the display device 30 can be designated. Therefore, as shown in FIG. 5, the coordinate position in the operation area OR of the operation surface AR of the touch pad 102 is associated with the coordinate position in the display screen 32 by the display device 30 on a one-to-one basis. The operation region OR may be provided on the entire operation surface AR. For example, the coordinate position P1 (x ₁ , y ₁ ) in the operation area OR (the origin is the upper left corner) of the touch pad 102 designated by the user with the finger 26 is the coordinate position P2 in the display screen 32 (the origin is the upper left corner). There is a one-to-one correspondence to (X ₁ , Y ₁ ). When the horizontal width of the operation area OR of the operation surface AR of the touch pad 102 is wx, the vertical width is wy, the horizontal width of the display screen 32 is WX, and the vertical width is WY, the following relationship is established.
_{x 1 / wx = X 1 /} WX ··· (1)
y ₁ / wy = Y ₁ / WY (2)

  Such an input device 100 outputs absolute position coordinates corresponding to the contact position on the operation surface AR as position information. The processing device 200 uses this position information to place a pointer as a position specifying mark at the coordinate position P2 in the display screen 32 of the display device 30 corresponding to the coordinate position P1 designated by the user with the finger 26 on the operation surface AR. Process to display.

  In FIG. 5, objects OBJ <b> 1 to OBJ <b> 4 that are icons are displayed in the display screen 32. The object OBJ1 is an icon for starting an application that realizes the camera photographing function. The object OBJ2 is an icon for starting the browser. The object OBJ3 is an icon for starting an application for reproducing a recorded voice message or an application for displaying a recorded text message. The object OBJ4 is an icon for starting an application that realizes a telephone function. The objects OBJ1 to OBJ4 are respectively positioned with pointers, and a corresponding application is activated by, for example, a click operation. Alternatively, the positions of the objects OBJ1 to OBJ4 are aligned with the pointers, and the display positions of the corresponding icons are moved by, for example, a drag operation.

  In the input device 100, a space is provided between the touch pad 102 and the bottom surface, and cushion materials 104 are arranged near the four corners of the touch pad 102 as shown in FIG. 104 has a structure supported by 104. Further, a push switch (switch in a broad sense) 106 is disposed in this space, and is disposed on the back side of the operation surface AR of the touch pad 102 by applying pressure to the operation surface AR of the touch pad 102. Further, the head of the push switch 106 is pressed. Information corresponding to the ON state or OFF state of the push switch 106 is output to the processing apparatus 200 as the press information described above.

  FIGS. 6A and 6B schematically show a cross-sectional structure along the line AA ′ in FIG. FIG. 6A schematically shows a state in which the user does not bring the finger 26 into contact with the operation surface AR. FIG. 6B schematically shows a state in which the user brings the finger 26 into contact with the operation surface AR and applies pressure to the operation surface AR. 6A and 6B, parts that are the same as those in FIGS. 4A and 4B are given the same reference numerals, and descriptions thereof will be omitted as appropriate.

  As shown in FIG. 6A, the input device 100 includes a touch pad 102 having an operation surface AR, a movable body 108 on which the touch pad 102 is disposed, and a press disposed on the back side of the movable body 108. And a switch 106. The press switch 106 can be pressed by applying pressure to the operation surface AR of the touch pad 102, and press information is output in accordance with the pressed state of the press switch 106. Further, the input device 100 moves the position of the movable body 108 to a predetermined pressing position of the pressing switch 106 in the pressed state, and returns the position of the movable body 108 to the initial position (predetermined position) in the non-pressed state. Alternatively, a plurality of cushion materials 104 are provided.

  In the present embodiment, the push switch 106 is a switch in which the difference between the initial position that is turned off when not pressed and the pressed position that is turned on when pressed is small. Further, in the present embodiment, the push switch 106 is arranged so that the head of the push switch 106 is pushed to the bottom surface that configures the space provided in the input device 100.

  Thus, when the finger 26 is in a non-contact state with the operation surface AR of the touch pad 102, the movable body 108 and the touch pad 102 are in the initial positions shown in FIG. When the finger 26 comes into contact with the operation surface AR of the touch pad 102 and further pressure is applied, for example, the cushion material 104 contracts, the movable body 108 and the touch pad 102 sink, and the back surface of the movable body 108 Is pressed.

  The configuration of the input device 100 in the present embodiment is not limited to the configuration described with reference to FIGS. 6 (A) and 6 (B). A switch may be provided on the end face side of the touch pad 102 as long as the movement of the touch pad 120 is reflected in the switching of the switch. That is, the input device 100 is configured to include a touch pad 102 having an operation surface AR and a switch arranged to detect the movement of the touch pad 102, and the switch is applied when pressure is applied to the operation surface AR. Are switched, and a pressed state or a non-pressed state is detected according to the switching state of the switch. At this time, the input device 100 moves the movable position of the touch pad 102 to the switch switching position in the pressed state, and returns the movable position of the touch pad 102 to the predetermined position in the non-pressed state. A cushion material 104 can be provided.

  As described above, the input device 100 according to this embodiment includes the position detection unit 110 that detects the contact position of the finger on the operation surface AR, and whether the input device 100 is in a pressed state or a non-pressed state in which a predetermined pressure is applied to the operation surface AR. And a press detection unit 120 for detecting. The input device 100 may have a configuration in which each of the position detection unit 110 and the press detection unit 120 detects a detection target independently of each other.

  Whether the user has pressed the touch pad 102 or not pressed can be recognized by the processing device 200 that has received the press information by changing the display on the display device 30.

  7A and 7B show examples of pointer display. FIG. 7A shows a display example of a pointer in a non-press state. FIG. 7B shows a display example of the pointer in the pressed state.

  Based on the press information from the input device 100, the processing device 200 displays the pointer PT1 displayed on the display screen 32 when the finger 26 is in contact with the operation surface AR in a non-pressed state, and when the finger 26 is in the pressed state in the pressed state. The pointer PT2 displayed on the display screen 32 is made different. For example, the processing device 200 performs processing for displaying the pointer PT1 on the display screen 32 smaller than the pointer PT2. Hereinafter, the display of the pointer PT1 is referred to as a touch display, and the display of the pointer PT2 is referred to as a press display. Instead of changing the sizes of the pointer PT1 and the pointer PT2, a completely different mark may be displayed.

Further, the processing device 200 has a ratio of the contact area (finger area φ9 mm) R1 of the user's finger 26 on the operation surface AR to the area (wx × wy) of the operation region OR of the operation surface AR in FIG. It is desirable to display the pointer so as to be larger than the display area R2 of the pointer with respect to the area of 32 (WX × WY). In this way, the display screen 32 can display a smaller area than the area touched by the finger 26 on the operation surface AR, so that the user can easily overlay the position of the pointer on a desired object or the like. Become.
R1 / (wx × wy)> R2 / (WX × WY) (3)

[Processing equipment]
The processing device 200 determines a pointing operation, a click operation, a long click operation, a drag operation, and a flick operation on the operation surface AR of the touch pad 102 included in the input device 100 described above. The processing device 200 determines a pointing operation based on touch coordinates that are positions where the finger 26 is in contact with the operation surface AR in a non-pressed state. Further, the processing device 200 determines that the touch operation is not performed after the transition from the non-press state to the press state and the click operation is performed when the touch coordinate is not changed in a predetermined time or less. In addition, the processing device 200 determines that the touch coordinate does not move after the transition from the non-press state to the press state and the long click operation is performed when a predetermined time or more has elapsed. In addition, the processing device 200 determines that the operation is a drag operation when the touch coordinate moves from the non-press state to the press state and then shifts to the non-press state. Note that the processing device 200 determines that the operation is a drag operation even when the touch coordinates change in the pressed state after the long click operation. Furthermore, the processing apparatus 200 determines that the flick operation is performed when the touch coordinates move quickly after the transition from the non-press state to the press state and then shifts to the non-press state. The processing device 200 that has determined the operation as described above performs processing corresponding to the determined operation, and reflects the processing result on the display screen 32 of the display device 30.

<Pointing operation>
The pointing operation in the present embodiment is an operation in which a pointer corresponding to the contact position of the user's finger 26 in the operation area OR is displayed on the display screen 32 of the display device 30 and the target is aligned with an object arranged on the display screen 32. .

  FIG. 8A, FIG. 8B, FIG. 8C, and FIG. 8D are explanatory diagrams of the pointing operation in this embodiment. Each of FIG. 8A to FIG. 8D represents the operation state of the user with respect to the input device 100 and the display screen 32 of the display device 30 corresponding to the operation state. 8A to 8D, the touch pad 102 is illustrated in a simplified manner, and FIGS. 6A, 6B, 7A, and 7B are illustrated. The same reference numerals are given to the same parts as in FIG.

  As shown in FIG. 8A, in the input device 100, when the user's finger 26 is not in contact with the operation surface of the touch pad 102, the input device 100 displays the position information corresponding to this state in the processing device 200. Output for. Further, the input device 100 outputs press information indicating that the operation surface is in a non-pressed state to the processing device 200. The processing device 200 performs processing for hiding the pointer on the display screen 32 of the display device 30.

  As shown in FIG. 8B, in the input device 100, when the user's finger 26 is lightly touching the operation surface of the touch pad 102, the input device 100 processes position information corresponding to the contact position of the finger 26. Output to the device 200. Further, the input device 100 outputs press information indicating that the operation surface is in a non-pressed state to the processing device 200. The processing device 200 performs processing for displaying the pointer PT on the display position corresponding to the contact position on the operation surface on the display screen 32 of the display device 30 by touch display. At this time, the processing device 200 performs a process of displaying the pointer PT through the object or background image that overlaps the pointer PT in the display screen 32. In the present embodiment, the user displays the pointer PT in the display area without looking at the operation source. This makes it possible to recognize an object or background image that originally existed at the position where the user displayed the pointer PT. This eliminates the need for the user to confirm an object or the like that overlaps the displayed pointer PT suddenly, thereby further improving the operability of the user.

  Subsequently, as illustrated in FIG. 8C, when the input device 100 is horizontally moved in a state where the finger 26 is lightly in contact with the operation surface of the touch pad 102, the input device 100 includes a plurality of fingers 26 on the movement path. Position information corresponding to each of the contact positions is output to the processing device 200. Further, the input device 100 outputs press information indicating that the operation surface is in a non-pressed state to the processing device 200 while the finger 26 is moved horizontally. The processing device 200 performs processing for displaying the pointer PT on the display screen 32 of the display device 30 at a display position corresponding to each contact position on the operation surface by a touch display. Thereby, the pointer PT is displayed so as to move in the display screen 32.

  Thereafter, as illustrated in FIG. 8D, when the finger 26 is released from the operation surface of the touch pad 102 in the input device 100, the input device 100 displays position information corresponding to a state in which the finger 26 is not in contact with the operation surface. Is output to the processing device 200. Further, the input device 100 outputs press information indicating that the operation surface is in a non-pressed state to the processing device 200. The processing device 200 performs processing for hiding the pointer on the display screen 32 of the display device 30.

  As described above, the processing device 200 performs a process of displaying a pointer at a display position in the display screen 32 of the display device 30 corresponding to the contact position of the finger 26 in the operation region as a pointing process corresponding to the pointing operation.

  As described above, when the finger 26 is in contact with the operation surface, the processing device 200 displays a pointer at the display position in the display screen 32 corresponding to the contact position, and the finger 26 is in a non-contact state with the operation surface. At this time, a process of hiding the pointer in the display screen 32 is performed. Thus, the pointer is displayed on the display screen 32 of the display device 30 corresponding to the contact position in the operation area of the touch pad 102 only while the finger 26 is in contact with the touch pad 102.

<Click operation>
The click operation in the present embodiment is performed by pressing the push switch 106 by applying pressure to the operation surface with the user's finger 26 to turn on the push switch 106 in a short time while maintaining the contact position of the finger 26 in the operation region. This is an operation to cancel the ON state of.

  FIG. 9A, FIG. 9B, FIG. 9C, FIG. 10A, and FIG. 10B are explanatory diagrams of the click operation in this embodiment. Each of FIGS. 9A to 9C, FIG. 10A, and FIG. 10B shows the operation state of the user with respect to the input device 100 and the display screen 32 of the display device 30 corresponding to this operation state. Are associated with each other. 9 (A) to 9 (C), FIG. 10 (A), and FIG. 10 (B), parts similar to those in FIGS. 8 (A) to 8 (D) are denoted by the same reference numerals, The description will be omitted as appropriate.

  As shown in FIG. 9A, in the input device 100, when the user's finger 26 is not in contact with the operation surface of the touch pad 102, the input device 100 displays the position information corresponding to this state in the processing device 200. Output for. Further, the input device 100 outputs press information indicating that the operation surface is in a non-pressed state to the processing device 200. The processing device 200 performs processing for hiding the pointer on the display screen 32 of the display device 30.

  As shown in FIG. 9B, in the input device 100, when the user's finger 26 is lightly touching the operation surface of the touch pad 102, the input device 100 processes position information corresponding to the contact position of the finger 26. Output to the device 200. Further, the input device 100 outputs press information indicating that the operation surface is in a non-pressed state to the processing device 200. The processing device 200 performs a process of displaying the pointer PT by a touch display at a display position corresponding to the contact position of the operation area on the display screen 32 of the display device 30. At this time, the processing device 200 performs a process of displaying the pointer PT through the object or background image that overlaps the pointer PT in the display screen 32.

  Subsequently, as shown in FIG. 9C, when pressure is applied to the operation surface of the touch pad 102 by the finger 26 in the input device 100, the touch pad 102 is lowered to the pressed position and is arranged on the back side of the touch pad 102. The push switch 106 is pressed. The input device 100 outputs position information corresponding to the contact position of the finger 26 in the operation area to the processing device 200. Further, the input device 100 outputs press information indicating that the operation surface is in a pressed state to the processing device 200. The processing device 200 performs a process of displaying the pointer PT on the display screen 32 of the display device 30 at a display position corresponding to the contact position of the operation area in a press display. Thereby, the pointer PT is changed from the touch display to the press display.

  Thereafter, as shown in FIG. 10A, when the pressure applied to the operation surface is weakened while the input device 100 is kept in contact without changing the contact position of the finger 26 in the operation region, the position of the push switch 106 is initialized. Return to position. At this time, the input device 100 outputs position information corresponding to the contact position of the finger 26 in the operation area to the processing device 200. Further, the input device 100 outputs press information indicating that the operation surface is in a non-pressed state to the processing device 200. The processing device 200 performs a process of displaying the pointer PT on the display screen 32 of the display device 30 at a display position corresponding to the contact position of the operation area by a touch display. As a result, the pointer PT changes from the press display to the touch display.

  Then, as shown in FIG. 10B, when the finger 26 is released from the operation surface of the touch pad 102 in the input device 100, the input device 100 has positional information corresponding to a state in which the finger 26 is not in contact with the operation surface. Is output to the processing device 200. Further, the input device 100 outputs press information indicating that the operation surface is in a non-pressed state to the processing device 200. The processing device 200 performs processing for hiding the pointer on the display screen 32 of the display device 30.

  Here, as the click process corresponding to the click operation, the processing device 200 starts the application corresponding to the object at the display position of the pointer PT when the press display is performed, and the numerical value by the spin button at the display position. Process to update the display.

  As described above, the pressing of the push switch 106 corresponds to the touch of the finger on the operation surface of the conventional touch pad, and the release of the press of the press switch 106 corresponds to the operation of releasing the finger from the operation surface of the conventional touch pad. .

  That is, the processing device 200 displays a pointer at a display position corresponding to the contact position in the display screen 32 when the finger 26 is in contact with the operation surface in the non-press state. Further, the processing device 200 can execute a click process corresponding to the click operation at a display position corresponding to the contact position in the display screen 32 in the press state.

<Long click operation>
In the long click operation in the present embodiment, the push switch 106 is pressed, and the touch position of the finger 26 in the operation area is maintained for a predetermined long click determination time or longer and the press of the push switch 106 is released at the contact position. It is. Since such a long click operation is the same as the above click operation, detailed description is omitted.

  In this case, the pressing of the push switch 106 corresponds to the finger touching the operation surface of the conventional touch pad, and the release of the pressing of the press switch 106 corresponds to the operation of releasing the finger from the operation surface of the conventional touch pad.

  That is, when the processing position is shifted to the non-press state after the contact position is maintained within the predetermined range in the press state for the long click determination period or longer, the processing device 200 is long at the display position in the display area corresponding to the contact position. A long click process corresponding to the click operation is executed.

<Drag operation>
In the drag operation in the present embodiment, the push switch 106 is pressed and the contact position of the finger 26 on the operation surface is maintained (fixed) for the first period or longer, and then the touch position of the finger 26 is moved and the push switch 106 is moved. This is an operation of releasing the finger 26 from the touch pad 102 at a position different from that when the button is pressed.

  FIG. 11A, FIG. 11B, FIG. 11C, FIG. 12A, and FIG. 12B are explanatory diagrams of the drag operation in this embodiment. Each of FIGS. 11A to 11C, 12A, and 12B shows a user operation state with respect to the input device 100 and a display screen 32 of the display device 30 corresponding to the operation state. Are associated with each other. In addition, in FIG. 11 (A)-FIG. 11 (C), FIG. 12 (A), and FIG. 12 (B), the same code | symbol is attached | subjected to the part similar to FIG. 8 (A)-FIG. 8 (D), The description will be omitted as appropriate.

  As shown in FIG. 11A, in the input device 100, when the user's finger 26 is not in contact with the operation surface of the touch pad 102, the input device 100 displays the position information corresponding to this state in the processing device 200. Output for. Further, the input device 100 outputs press information indicating that the operation surface is in a non-pressed state to the processing device 200. The processing device 200 performs processing for hiding the pointer on the display screen 32 of the display device 30.

  As shown in FIG. 11B, in the input device 100, when the user's finger 26 is lightly touching the operation surface of the touch pad 102, the input device 100 processes position information corresponding to the contact position of the finger 26. Output to the device 200. Further, the input device 100 outputs press information indicating that the operation surface is in a non-pressed state to the processing device 200. The processing device 200 performs a process of displaying the pointer PT by a touch display at a display position corresponding to the contact position of the operation area on the display screen 32 of the display device 30. At this time, the processing device 200 performs a process of displaying the pointer PT through the object or background image that overlaps the pointer PT in the display screen 32. Here, it is assumed that the display position of the pointer PT is at the position of an object that can be dragged.

  Subsequently, as illustrated in FIG. 11C, when pressure is applied to the operation surface of the touch pad 102 with the finger 26 in the input device 100, the touch pad 102 is lowered to the pressed position and disposed on the back side of the touch pad 102. The push switch 106 is pressed. The input device 100 outputs position information corresponding to the contact position of the finger 26 in the operation area to the processing device 200. Further, the input device 100 outputs press information indicating that the operation surface is in a pressed state to the processing device 200. The processing device 200 performs a process of displaying the pointer PT in a press display at a display position corresponding to the contact position of the operation area on the display screen 32 of the display device 30. Thereby, the pointer PT is changed from the touch display to the press display. In FIG. 11B and FIG. 11C, since the pointer PT is transparently displayed by touching, the user can operate while recognizing that the pointer PT overlaps the object OBJ3 (see FIG. 5).

  Here, while applying pressure to the operation surface, the operation waits while maintaining the contact position of the finger 26 for the first period or longer. As a result, the object at the position of the pointer PT is displayed in a dragged state, and a drag process corresponding to the drag operation is started.

  Thereafter, as shown in FIG. 12A, the contact position of the finger 26 in the operation region is horizontally moved while applying pressure to the operation surface in the input device 100. At this time, the input device 100 outputs position information corresponding to each of a plurality of contact positions on the movement path of the finger 26 to the processing device 200. Further, the input device 100 outputs press information indicating that the operation surface is in a pressed state to the processing device 200 while the finger 26 is moved horizontally. The processing device 200 displays the pointer PT in the press display on the display screen 32 of the display device 30 corresponding to the contact position of the operation area, and causes the dragged object to follow the pointer PT in the dragged state. Process to display. Thus, the pointer PT moves in the display screen 32 while being pressed, and the corresponding object is dragged.

  Then, as shown in FIG. 12B, when the finger 26 is released from the operation surface of the touch pad 102 in the input device 100, the input device 100 displays position information corresponding to a state in which the finger 26 is not in contact with the operation surface. Is output to the processing device 200. Further, the input device 100 outputs press information indicating that the operation surface is in a non-pressed state to the processing device 200. The processing device 200 performs processing for hiding the pointer on the display screen 32 of the display device 30. The object moved by dragging is returned to the normal display at the display position in the display screen 32 corresponding to the contact position that is not in contact with the operation surface, and the drag process is terminated.

  Here, the processing apparatus 200 performs a process of moving an object (such as an icon) at the position of the pointer PT and a process of sucrose the entire screen as a drag process corresponding to the drag operation.

  As described above, the pressing of the push switch 106 corresponds to the touch of the finger on the operation surface of the conventional touch pad, and the release of the press of the press switch 106 corresponds to the operation of releasing the finger from the operation surface of the conventional touch pad. .

  That is, it is assumed that the finger 26 is moved from the first contact position, which has been shifted to the pressed state on the operation surface, to the second contact position while being pressed, and has shifted to the non-pressed state at the second contact position. At this time, the processing device 200 performs a drag process corresponding to a drag operation determined by the first contact position and the second contact position. This drag process is performed on the object at the display position in the display screen 32 where the contact position corresponds to the first contact position in the pressed state. Then, the processing device 200 ends the drag process at the display position in the display screen 32 corresponding to the second contact position that is the contact position when shifting from the pressed state to the non-pressed state.

<Flick operation>
In the flick operation in the present embodiment, the pressing switch 106 is pressed, the contact position of the finger 26 on the operation surface is moved in a short time, and the finger is touched from the touch pad 102 at a contact position different from the position where the press switch 106 is pressed. This is an operation of releasing 26. The procedure of such a flick operation is almost the same as the procedure of the drag operation. Specifically, after the finger 26 is maintained at the first contact position shifted to the pressed state on the operation surface for a predetermined period or longer, the finger 26 is moved to the second contact position in the pressed state, and at the second contact position, Assume that the state has shifted to a non-press state. At this time, the processing apparatus 200 performs a flick process corresponding to a flick operation determined by the first contact position and the second contact position. More specifically, the drag process and the flick process are distinguished and executed based on the amount of movement per unit time from the first contact position to the second contact position. Therefore, detailed description of the flick operation is omitted.

  Here, the processing device 200 performs a process of scrolling the screen in the finger movement direction and a process of switching the screen as the flick process corresponding to the flick operation.

  In this case, the pressing of the push switch 106 corresponds to the finger touching the operation surface of the conventional touch pad, and the release of the pressing of the press switch 106 corresponds to the operation of releasing the finger from the operation surface of the conventional touch pad.

  That is, when the amount of movement per unit time from the first contact position to the second contact position is equal to or greater than a given threshold, the processing device 200 performs a flick process. On the other hand, when the amount of movement per unit time is less than the threshold value, the processing device 200 performs a drag process.

  Next, a configuration example for realizing the functions of the processing device 200 described above will be described.

  FIG. 13 shows a functional block diagram of a configuration example of the processing apparatus 200 of FIG. In FIG. 13, the same parts as those in FIG.

  The processing device 200 includes a contact determination unit 210, a position detection processing unit 220, a press determination unit 230, an operation determination unit 240, a process execution unit 260, and a display processing unit 280. The operation determination unit 240 includes a click operation determination unit 242, a long click operation determination unit 244, a drag operation determination unit 246, and a flick operation determination unit 248. The process execution unit 260 includes a click processing unit 262, a long click processing unit 264, a drag processing unit 266, and a flick processing unit 268.

  The contact determination unit 210 determines whether the finger 26 is in contact with the operation surface of the input device 100 based on position information from the input device 100 or in a non-contact state. For example, by determining in advance the position information output by the input device 100 in the non-contact state, the contact determination unit 210 determines whether the contact state or the non-contact state based on the position information from the input device 100. can do. The position detection processing unit 220 performs processing for detecting the coordinates of the contact position of the finger 26 in the operation area based on the position information from the input device 100.

  Based on the press information from the input device 100, the press determination unit 230 determines whether or not a predetermined pressure or more is applied to the operation surface, the touch pad 102 is lowered to the pressed position, and the press switch 106 is pressed. The push switch 106 is turned on when pressed, and is turned off when the push switch 106 is released. The press determination unit 230 determines whether or not the press switch 106 is in the pressed state based on whether or not the press switch 106 is in the ON state. When the press switch 106 is in the ON state, the press determination unit 230 determines the press state. When in the state, it is determined as a non-press state.

  Based on the position information and the press information from the input device 100, the operation determination unit 240 is selected from the above pointing operation, click operation, long click operation, drag operation, and flick operation. Determine which operation is being performed.

  Specifically, the click operation determination unit 242 determines whether the click operation described with reference to FIGS. 9A to 9C, 10 </ b> A, and 10 </ b> B is performed by the input device 100. Is determined based on the position information and the press information. The long click operation determination unit 244 determines whether the long click operation is described above based on position information and press information from the input device 100. The drag operation determination unit 246 determines whether or not the drag operation has been described with reference to FIGS. 11A to 11C, FIG. 12A, and FIG. Determine based on press information. The flick operation determination unit 248 determines whether the flick operation is described above based on the position information and the press information from the input device 100.

  The process execution unit 260 executes a process corresponding to the operation determined by the operation determination unit 240. Specifically, when the operation by the user's finger 26 is determined to be a click operation by the click operation determination unit 242, the click processing unit 262 executes the click process corresponding to the click operation. When the operation by the user's finger 26 is determined to be a long click operation by the long click operation determination unit 244, the long click processing unit 264 performs the long click process corresponding to the long click operation. When the operation by the user's finger 26 is determined to be a drag operation by the drag operation determination unit 246, the drag processing unit 266 performs the drag process corresponding to the drag operation. When the operation by the user's finger 26 is determined to be a flick operation by the flick operation determination unit 248, the flick processing unit 268 executes the flick process corresponding to the flick operation.

  The display processing unit 280 performs processing for generating an image corresponding to the processing result of each unit constituting the processing execution unit 260 and displaying the image on the display device 30. For the pointing operation, the processing device 200 obtains the coordinates of the display position on the display screen 32 corresponding to the coordinates of the contact position in the operation area detected by the position detection processing unit 220, for example, and the display processing unit 280 displays the display position coordinates. Performs processing to display a pointer at the position.

The processing apparatus 200 having the functions of the respective units in FIG. 13 is realized by the following configuration, for example.
FIG. 14 shows a hardware configuration example of the processing apparatus 200 of FIG. In FIG. 14, the same parts as those in FIG. 1, FIG. 6 (A) or FIG. 6 (B) are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  In addition to the input device 100 and the processing device 200, the information input device 20 can include a battery 320, a USB (Universal Serial Bus) socket 322, and a display LED (Light Emitting Diode) 324. The battery 320 supplies power to each unit of the input device 100 and the processing device 200. The battery 320 may be provided outside the information input device 20. A USB cable is connected to the USB socket 322, and data is transmitted / received to / from the outside under the control of the processing device 200. The display LED 324 performs display corresponding to the operation state of the input device 100 or the processing device 200.

  The processing device 200 includes a central processing unit (hereinafter referred to as CPU) 302, a read only memory (hereinafter referred to as ROM) 304, and a random access memory (hereinafter referred to as RAM). Prepare. Further, the processing apparatus 200 includes an interface (InterFace: hereinafter referred to as I / F) circuit 306 and a bus 308. The CPU 302, ROM 304, RAM 306, and I / F circuit 306 are connected via a bus 308.

  The CPU 300 reads a program or data stored in the ROM 302 or RAM 304 and executes a process corresponding to the read program or data. The ROM 302 stores a program executed by the CPU 300. A program executed by the CPU 300 is stored in the RAM 304, or a work area for the CPU 300 is secured. The I / F circuit 306 performs an interface process for receiving position information and press information from the input device 100. Further, the I / F circuit 306 performs interface processing for transmitting image data and display control signals corresponding to the display image displayed on the display device 30 to the display device 30.

  In such a configuration, the CPU 300 can realize the functions of each unit in FIG. 13 by software processing by executing processing corresponding to the program read from the ROM 302 or RAM 304. For example, the CPU 300 can realize the detection of the coordinates of the contact position of the finger in the operation area based on the position information from the input device 100 received via the I / F circuit 306 by software processing. Alternatively, the CPU 300 can detect whether the operation surface is in a pressed state or a non-pressed state by software processing based on the press information from the input device 100 received via the I / F circuit 306. Further, the CPU 300 outputs image data and display control signals corresponding to the processing results of the respective units to the display device 30 via the I / F circuit 306. By doing so, it is possible to display a pointer at a desired position on the display screen of the display device 30 and to display a processing result corresponding to the above operation on the operation surface on the display screen.

  15 and 16 are flowcharts of processing examples of the processing apparatus 200. FIG. For example, a program for executing the steps of FIGS. 15 and 16 is stored in advance in the ROM 302 or the RAM 304 of FIG. 14, and the CPU 300 reads the program and executes a process corresponding to the program.

  First, in the contact determination unit 210, the processing device 200 repeatedly determines whether or not the operating user's finger touches the operation surface of the input device 100 (step S100: N). When it is determined in step S100 that the finger has touched the operation surface (step S100: Y), the processing device 200 performs processing for detecting the coordinates of the contact position of the finger in the operation region in the position detection processing unit 220. Perform (step S102).

Subsequently, in the processing device 200, the position detection processing unit 220 uses the coordinates (x ₁ , y ₁ ) of the contact position of the finger in the operation area detected in step S 102 as the coordinates of the display screen of the display device 30 ( X ₁ , Y ₁ ) (Step S104). In step S104, since the aspect ratio of the operation area is different from the aspect ratio of the display screen, the coordinates of the display screen are converted by the following equation, for example.
X ₁ = x ₁ × (WX / wx) (4)
Y ₁ = y ₁ × (WY / wy) (5)

  In Expression (4) and Expression (5), the horizontal width of the operation area is wx, the vertical width is wy, the horizontal width of the display screen is WX, and the vertical width is WY.

  Thereafter, the processing device 200 causes the display processing unit 280 to display a pointer on the coordinates converted in step S104 by touch display (step S106).

  Next, in the press determination unit, the processing device 200 determines whether or not the push switch 106 disposed on the back side of the operation surface of the input device 100 has been pressed (step S108). When it is determined in step S108 that the push switch 106 is not pressed (step S108: N), the processing device 200 returns to step S102. On the other hand, when it is determined in step S108 that the push switch 106 has been pressed (step S108: Y), the processing device 200 changes the pointer on the display screen to press display in the display processing unit 280 (step S110).

Subsequently, in the position detection processing unit 220, the processing device 200 performs a process of detecting the coordinates of the contact position of the finger in the operation area (step S112). Then, the processing device 200 converts the coordinates of the touch position of the finger in the operation area detected in step S112 into (x ₁ , y ₁ ) on the display screen of the display device 30 (X ₁ , Y ₁ ). (Step S114). Step S114 is the same as step S104.

  Thereafter, in the display processing unit 280, the processing device 200 continues to display the pointer on the coordinates converted in step S114 in a press display (step S116).

  Next, in the operation determination unit 240, the processing device 200 determines whether or not the coordinates corresponding to the finger contact position in the operation area have changed compared to the coordinates at the previous detection (step S118). When it is determined in step S118 that the coordinate has changed (step S118: Y), the processing device 200 uses the operation determination unit 240 to provide a coordinate change amount (a movement amount of the contact position) per unit time. It is determined whether or not the threshold value is exceeded (step S120). Step S120 is performed by the drag operation determination unit 246 or the flick operation determination unit 248, for example.

  When it is determined in step S120 that the amount of change in coordinates per unit time is greater than or equal to the threshold (step S120: Y), the processing device 200 causes the flick processing unit 268 to perform the flick process corresponding to the flick operation. This is performed (step S122). In step S122, a flick process is performed according to the contact position (coordinates) before and after the movement in the operation area (for example, the moving distance and moving direction of the contact position).

  When it is determined in step S120 that the amount of change in coordinates per unit time is less than the threshold (step S120: N), the processing device 200 causes the drag processing unit 266 to perform the above-described drag processing corresponding to the drag operation. This is performed (step S124). In step S124, a drag process is performed according to the contact position (coordinates) before and after the movement in the operation area (for example, the moving distance and moving direction of the contact position).

  When it is determined in step S118 that the coordinates have not changed (step S118: N), the processing device 200 determines in the operation determination unit 240 whether or not a predetermined time has elapsed since the pressing of the pressing switch 106. (Step S126). Step S126 is performed by, for example, the click operation determination unit 242 or the long click operation determination unit 244.

  When it is determined in step S126 that a predetermined time or more has elapsed since the pressing of the push switch 106 (step S126: Y), the processing device 200 determines in the long click operation determination unit 244 whether the long click operation is being performed. (Step S128). When it is determined in step S128 that the long click operation is not being performed (step S128: N), the processing device 200 starts the long click process corresponding to the long click operation in the long click processing unit 264 (step S140). ), The process returns to step S112. Thereby, the long click process is continued.

  When it is determined in step S126 that the predetermined time or more has not elapsed (step S126: N), the processing device 200 determines whether or not the pressing of the press switch 106 is released in the click operation determination unit 242. (Step S130). When it is determined in step S128 that the long click operation is being performed (step S128: Y), the processing device 200 determines whether or not the press of the push switch 106 is released in the click operation determination unit 242 (step S128: Y). Step S130).

  When it is determined in step S130 that the press of the push switch 106 has been released (step S130: Y), the processing device 200 determines in the click operation determination unit 242 whether or not a long click operation is being performed (step S132). .

  When it is determined in step S132 that the long click operation is being performed (step S132: Y), the processing device 200 ends the long click process performed in the long click processing unit 264 (step S134). When it is determined in step S132 that the long click operation is not being performed (step S132: N), the processing device 200 performs the above click process corresponding to the click operation in the click processing unit 262 (step S136).

  When it is determined in step S130 that the press of the push switch 106 has not been released (step S130: N), the processing device 200 returns to step S112 and continues the processing.

  Subsequent to step S134 or step S136, the processing device 200 changes the pointer on the display screen to the touch display in the display processing unit 280 (step S138).

  Subsequent to step S122, step S124, or step S138, the processing device 200 determines in the contact determination unit 210 whether or not the finger is not in contact with the operation surface of the input device 100 (step S142).

  When it is determined in step S142 that the finger is not in contact with the operation surface (step S142: N), the processing device 200 returns to step S102 and continues the process.

  On the other hand, when it is determined in step S142 that the finger is not in contact with the operation surface (step S142: Y), the processing device 200 performs a process in the display processing unit 280 to hide the pointer on the display screen. This is performed (step S144). Thereby, the processing apparatus 200 ends a series of processing (end).

  FIG. 17 shows a flowchart of a processing example of the drag processing in step S124 of FIG. For example, a program for executing each step of FIG. 17 is stored in advance in the ROM 302 or RAM 304 of FIG. 14, and the CPU 300 reads the program and executes processing corresponding to the program.

In the position detection processing unit 220, the processing device 200 performs processing for detecting the coordinates of the contact position of the finger within the operation region (step S150). Next, the processing device 200 uses the position detection processing unit 220 to display the coordinates of the contact position of the finger in the operation area detected in step S150 as (x ₁ , y ₁ ) and the coordinates of the display screen of the display device 30 ( X ₁ , Y ₁ ) (step S152). Step S152 is the same as step S104.

  Thereafter, in the display processing unit 280, the processing device 200 continues to display the pointer on the coordinates converted in step S152 by press display (step S154).

  Subsequently, in the drag processing unit 266, the processing device 200 performs a drag process according to the coordinates converted in step S152 (step S156). For example, when an object exists at the coordinates, a process of displaying the object while dragging is performed.

  Then, the processing device 200 determines in the drag operation determination unit 246 whether or not the pressing of the pressing switch 106 has been released (step S158).

  When it is determined that the pressing of the push switch 106 has not been released (step S158: N), the processing device 200 returns to step S150 and continues the processing. On the other hand, when it is determined in step S158 that the pressing switch 106 has been released (step S158: Y), the processing device 200 ends the drag process (END).

  FIG. 18 shows a flowchart of a processing example of the flick processing in step S122 of FIG. For example, the ROM 302 or the RAM 304 in FIG. 14 stores a program for executing each step in FIG. 18 in advance, and the CPU 300 reads the program and executes a process corresponding to the program.

In the position detection processing unit 220, the processing device 200 performs a process of detecting the coordinates of the contact position of the finger in the operation area (step S160). Next, the processing device 200 uses the position detection processing unit 220 to calculate the coordinates of the contact position of the finger in the operation area detected in step S160 as (x ₁ , y ₁ ) and the coordinates of the display screen of the display device 30 ( X ₁ , Y ₁ ) (step S162). Step S162 is the same as step S104.

  Thereafter, in the display processing unit 280, the processing device 200 continues to display the pointer on the coordinates converted in step S162 by press display (step S164).

  Subsequently, the processing device 200 performs a flick process in the flick processing unit 268 according to the coordinates converted in step S162 (step S166). For example, a process of scrolling the screen in the finger movement direction and a process of switching the screen are performed.

  Then, the processing device 200 determines in the flick operation determination unit 248 whether the pressing of the pressing switch 106 has been released (step S168).

  When it is determined that the push switch 106 has not been released (step S168: N), the processing device 200 returns to step S160 and continues the process. On the other hand, when it is determined in step S168 that the press of the push switch 106 has been released (step S168: Y), the processing device 200 ends the flick process (end).

Note that the processing device 200 has an ASIC (Application Specific
Integrated Circuit) or dedicated hardware. In this case, in the processing device 200, processing corresponding to each step of FIGS. 15 to 18 is executed in each unit configured by hardware.

  As described above, according to this embodiment, when the operating body is in contact with the operation surface of the input device 100, the pointer is displayed at the display position corresponding to the contact position in the display screen 32 of the display device 30. Can do. As a result, the touch on the operation surface is not handled as a click operation as in the conventional case, and the user who operates can be informed of the position of the operation body before the click operation. Further, by applying pressure to the operation surface of the input device 100 with the operating body, a confirmation instruction at the display position corresponding to the contact position in the display screen is performed. As a result, the input of information can be accepted by distinguishing the contact of the operating body on the operation surface and the confirmation instruction, and the user can see the operation source even when the operation surface and the display screen are separated. It is possible to input a variety of information.

[Others]
In the present embodiment, the display device 30 is an HMD, and a display device that is difficult to look at the operation source is described as an example. However, the display device according to the present embodiment is limited to the HMD. is not.

  FIG. 19 schematically shows another example of an image in which the user uses the information processing apparatus of FIG. FIG. 19 illustrates an example in which the display device applied to the information processing apparatus in the present embodiment is a flat display device. In FIG. 19, the same parts as those in FIG. 2 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  The information processing device shown in FIG. 19 is different from the information processing device shown in FIG. 2 in that the display device 30 is replaced with a flat display device as the display device 30a, and between the display device 30a and the information input device 20a. Is connected through a wireless signal transmission path. Examples of the flat display device include a liquid crystal display device, a plasma display device, and an organic electroluminescence (EL) display device. The information input device 20a can perform the same processing as the information input device 20 except that display control can be performed on the display device 30a via a wireless signal transmission path.

  In this case, operation information corresponding to the operation by the user 24 is input to the information input device 20a. The display device 30a displays a result reflecting the operation information by displaying an image corresponding to the operation information input to the information input device 20a.

  As described above, even in the configuration as shown in FIG. 19, the user 24 can input a variety of information by pointing operation on the operation surface without looking at the operation source.

  The present invention is not limited to the above-described embodiment, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

  (1) In the above embodiment, the input device according to the present invention will be described by taking as an example an apparatus having the configuration shown in FIGS. 4 (A), 4 (B), 6 (A), and 6 (B). However, the present invention is not limited to the configuration of the input device.

  (2) In the above embodiment, the display device according to the present invention has been described with reference to the example shown in FIG. 2 or FIG. 19, but the present invention is not limited to the form of the display device. As the display device according to the present invention, a display device of another aspect such as a projector may be adopted.

  (3) In the above embodiment, the processing example corresponding to each operation of the pointing operation, the click operation, the long click operation, the drag operation, and the flick operation has been described, but the present invention is a process corresponding to each of the above operations. It is not limited to the contents of.

  (4) In the above embodiment, the pointing operation, the click operation, the long click operation, the drag operation, and the flick operation are described as examples of the operation on the operation surface (operation area) of the input device. It is not limited to the operation of.

  (5) In the above embodiment, the user's finger has been described as an example of the operation body for the operation surface (operation area), but the present invention is not limited to the type of operation body.

  (6) The present invention can be realized in various modes. For example, the present invention can be realized in the form of an information input method and information processing method using an input device having an operation surface according to the present invention, a computer program that realizes these methods, a recording medium that records the computer program, and the like.

DESCRIPTION OF SYMBOLS 10 ... Information processing apparatus 20, 20a ... Information input device 22 ... Signal transmission path,
24 ... User, 26 ... Finger (operating body), 30 ... Display device (HMD),
30a ... display device (flat display device), 32 ... display screen (display area),
100 ... Input device 102 ... Touch pad 104 ... Cushion material
106 ... push switch (switch), 108 ... movable body, 110 ... position detector,
120 ... Press detection unit, 200 ... Processing device, 210 ... Contact determination unit,
220 ... position detection processing unit, 230 ... press determination unit, 240 ... operation determination unit,
242 ... Click operation determination unit, 244 ... Long click operation determination unit,
246 ... drag operation determination unit, 248 ... flick operation determination unit, 260 ... process execution unit,
262 ... Click processing unit, 264 ... Long click processing unit, 266 ... Drag processing unit,
268 ... Flick processing unit, 280 ... Display processing unit, 300 ... CPU,
302 ... ROM, 304 ... RAM, 306 ... I / F circuit, 308 ... Bus,
320 ... Battery, 322 ... USB socket, 324 ... Display LED,
AR ... operation surface, OBJ1 to OBJ4 ... object, OR ... operation area,
P1 ... coordinate position in the operation area, P2 ... coordinate position in the display area (display screen),
wx: horizontal width of the operation area, wy: vertical width of the operation area,
WX: Horizontal width of the display area (display screen), WY: Vertical width of the display area (display screen),
x ₁ ... x-coordinate of the coordinate position P1 in the operation area,
y ₁ ... y-coordinate of the coordinate position P1 in the operation area,
X ₁ ... x coordinate of the coordinate position P2 in the display area, Y ₁ ... y coordinate of the coordinate position P2 in the display area

Claims (11)

Coordinate position of the operation area is associated on a one-to-one to the coordinate position of the display area by the display device includes the display operation surface that provided separately from the display area by the device, the operation body in the operation surface An input device for detecting whether the press position or the non-press state in which a predetermined pressure or more is applied to the contact position and the operation surface;
A processing device that performs processing corresponding to the operation of the operating body on the operation surface,
The processor is
When the operation body is in contact with the operation surface in the non-pressed state, a position specifying mark is displayed in a first display mode at a display position corresponding to the contact position in the display area ,
When the operation body shifts to the pressed state on the operation surface, the position specifying mark is displayed in a second display mode different from the first display mode,
When the first contact position that has been shifted to the pressed state is moved to the second contact position in the pressed state, and when the second contact position has shifted to the non-pressed state, the first contact position and the An information processing apparatus that distinguishes and executes a drag process corresponding to a drag operation determined by a second contact position and a flick process corresponding to a flick operation. In claim 1,
The processor is
When the amount of movement per unit time from the first contact position to the second contact position is equal to or greater than a given threshold, the flick process is performed,
The information processing apparatus, wherein the drag process is performed when the movement amount per unit time is less than the threshold value. In any one of Claims 1 thru | or 2.
The processor is
When shifting from the non-pressed state to the pressed state, the position specifying mark is displayed in the second display mode, and after the contact position is maintained within a predetermined range, the state is shifted to the non-pressed state. ,
An information processing apparatus that performs a click process corresponding to a click operation at a display position of the position specifying mark, and displays the position specifying mark in the first display mode . In claim 3,
The processor is
After the contact position is maintained for a long click determination period or more within a predetermined range in the pressed state, and when the non-pressed state is entered,
An information processing apparatus , wherein a long click process corresponding to a long click operation is executed at a display position of the position specifying mark, and the position specifying mark is displayed in the first display mode . In any one of Claims 1 thru | or 4,
The processor is
An information processing apparatus, wherein when the operation body is in a non-contact state with the operation surface, a process for hiding the position specifying mark in the display area is performed. In any one of Claims 1 thru | or 5,
The processor is
The information processing apparatus displays the position specifying mark so that a ratio of a contact area of the operating body to an area of the operation area is larger than a display area of the position specifying mark with respect to the area of the display area. . In any one of Claims 1 thru | or 6 .
The processor is
An information processing apparatus that performs processing for transmitting the object or background image that overlaps the position specifying mark in the display area and displaying the position specifying mark. In any one of Claims 1 thru | or 7 ,
The input device is:
A touchpad having the operation surface;
A switch arranged to detect movement of the touchpad;
An information processing apparatus, wherein the switch is switched when the pressure is applied to the operation surface, and the pressed state or the non-pressed state is detected according to a switching state of the switch. In claim 8 ,
The input device is:
Including one or a plurality of cushion materials that move the movable position of the touch pad to the switching position of the switch in the pressed state and return the movable position of the touch pad to a predetermined position in the non-pressed state. An information processing apparatus characterized by the above. In any one of Claims 1 thru | or 9 ,
An information processing apparatus comprising the display device in which a processing result by the processing apparatus is reflected. In claim 10 ,
The display device
An information processing apparatus, which is a head-mounted display device.