JP5819477B2 - Information processing apparatus, information processing apparatus control method, control program, and recording medium - Google Patents

Description

  The present invention displays an image on a display device, operates in accordance with an operation signal received from an input device that detects an input operation on the display surface of the display device, and detects the approach of a finger or the like to the display surface of the display device. The present invention relates to an information processing apparatus that displays objects that can be selected by a user on the display surface.

  Since an intuitive operation while viewing a displayed image is possible, a display / input device that displays an image and receives an input operation on a display surface, a so-called touch panel, is widely used. The touch panel is connected to an information processing device such as a personal computer and used as a display device / input device of the information processing device.

  That is, the information processing apparatus connected to the touch panel displays an icon or the like on the touch panel, and when the touch panel detects a touch operation on the display surface, the touched icon is displayed from the touched position on the display surface. Identify and execute the process assigned to the icon.

  A general information processing apparatus displays icons in a fixed position at a predetermined position. However, in this configuration, a certain area of the display surface is occupied by the icons, and the work area becomes narrow. There are also cases where it is difficult for the user to recognize information other than icons.

  This will be described with reference to FIG. FIG. 10 is a diagram illustrating a related art, and is a diagram illustrating an example of a screen displaying a tool palette including a plurality of icons. As shown in the figure, the tool palette occupies a certain area on the screen, so that the area of the user's work area is reduced. In addition, characters and the like displayed behind the tool palette are not recognized by the user, and when the tool palette and other icons are displayed in a superimposed manner, the icon displayed behind the tool palette cannot be selected.

  Therefore, in Patent Document 1 below, an icon is displayed at the timing when the user's finger approaches the display surface. According to the configuration of Patent Literature 1, since the icon is not displayed while the user does not bring the finger close to the display surface, the user can easily recognize information other than the icon.

JP 2002-358162 A (released on December 13, 2002)

  However, the conventional technology as described above has a problem in that it does not consider a case where the user brings a finger close to the display surface for an operation other than selecting an icon (an icon displayed when the finger approaches). .

  For example, it is conceivable that the user brings his / her finger close to input handwritten characters on the display surface. In such a case, displaying an icon hinders the user's handwriting input. In addition, for example, it is conceivable that the user approaches a finger to select an icon that is already displayed on the display surface. In such a case, the icon displayed at the timing when the finger is brought closes the user operation. .

  In addition, what interferes with the user operation is not limited to an icon as long as it is an object to be selected by the user. For example, even a character or symbol causes the same problem. Further, the pointing object approaching the display surface is not limited to a finger, and the same problem occurs even with a touch pen or the like.

  The present invention has been made in view of the above problems, and its purpose is to display an object to be selected by the user when the pointing object approaches the display surface, and the user selects the object. An object of the present invention is to realize an information processing apparatus or the like that prevents the object from interfering with the user operation when not.

  In order to solve the above problems, an information processing apparatus according to the present invention displays an image on a display device and receives an operation signal received from an input device that detects an input operation performed on the display surface of the display device. An information processing apparatus that operates in accordance with the display object when the pointing object that the user has approached the display surface to perform an input operation on the display surface approaches the display surface to a predetermined display distance, Object display means for causing the display device to display objects that can be selected by the user on the display surface, and an approach for determining whether or not the pointing object is moving toward the object displayed by the object display means Determination means, and the object display means is configured such that the approach determination means does not move the pointing object toward the object. When it is determined, it is characterized in that to end the display of the object.

  Further, in order to solve the above-described problem, the control method of the information processing apparatus according to the present invention displays an image on a display device and detects an input operation performed on the display surface of the display device. A method of controlling an information processing apparatus that operates according to a received operation signal, wherein an indication object that is brought close to the display surface for a user to perform an input operation on the display surface is a predetermined display on the display surface An object display step for causing the display device to display an object that can be selected on the display surface when the user approaches the distance, and the pointing object moves toward the object displayed in the object display step. In the approach determining step for determining whether or not there is the approach determining step, the pointing object moves toward the object. When it is determined that there is no, it is characterized in that it comprises a display end step for ending the display of the object.

  According to the above configuration, the object is displayed when the pointing object approaches the display distance. When it is determined that the pointing object is not facing the object, the display of the object is terminated. The pointing object may be anything that can perform an input operation on the display surface, and may be, for example, a finger or a touch pen.

  Therefore, the user can recognize selectable objects by bringing the pointing object closer to the display distance. When it is determined that it is not necessary to select the recognized object, the pointing object is not moved toward the object (for example, an input operation is performed on an area on the display surface where the object is not displayed). The display of the object can be terminated.

  That is, according to the above configuration, when the pointing object is not close to the display distance, other images displayed on the display surface are not blocked by the object, and after the object is displayed, the user moves the object. If not selected, the object will not interfere with the user operation.

  Further, the approach determination means displays the straight line connecting the shortest distance between the pointing object and the display surface when the pointing object approaches the display surface to an erasure determination distance shorter than the display distance. When the intersection with the surface is not included in the area where the object is displayed, it is preferable to determine that the pointing object has not moved toward the object.

  Here, when the object is selected by the pointing object, the pointing object finally moves onto the object. For this reason, when the pointing object further approaches the display surface from the display distance, the intersection of the straight line connecting the shortest distance between the pointing object and the display surface is included in the area where the object is displayed. Become.

  Therefore, according to the above configuration, when the pointing object approaches the display surface to the erasure determination distance, the intersection of the straight line connecting the minimum distance between the pointing object and the display surface is the display of the object. In the case where it is not included in the indicated area, it is determined that the pointing object has not moved toward the object. Thereby, it is possible to determine whether or not the pointing object is moving toward the object with a simple configuration.

  Further, when the approach determination unit determines that the pointing object is moving toward the object, the information processing apparatus detects the object when the object is selected by the pointing object. It is preferable that a process execution means for executing a process assigned in advance is provided.

  According to the above configuration, when it is determined that the pointing object is moving toward the object, when the object is selected by the pointing object, the process assigned in advance to the object is executed.

  Therefore, after the object is displayed, the user moves the pointing object toward the displayed object, and selects the object with the pointing object, thereby causing the information processing apparatus to execute the process assigned to the object. be able to.

  In addition, the object display means may be configured in advance from an intersection of a straight line connecting the shortest distance between the pointing object and the display surface when the pointing object approaches the display surface to the display distance. It is preferable to display the object at a position separated by a predetermined distance.

  According to the above configuration, the object is displayed at a position away from the intersection of the straight line connecting the shortest distance between the pointing object and the display surface with the display surface by a predetermined distance. In other words, the object is not displayed at the position of the intersection of the straight line connecting the shortest distance between the pointing object and the display surface.

  As a result, when the user intends to bring the pointing object closer to a part on the display surface and the pointing object is brought closer to the display surface, the object is displayed on that part, and the object is not intended. Can be prevented from being selected.

  In addition, it is preferable to set the distance from the said intersection so that the user who made the pointing object approach to the display distance can easily select the object. In other words, the object is preferably displayed in the vicinity of the intersection.

  The object display means preferably displays a plurality of the objects so as to surround the intersection.

  According to said structure, a some object is displayed so that the intersection of this display surface of the straight line which connects the shortest distance of a pointing object and a display surface may be enclosed.

  As a result, a plurality of objects can be presented to the user without impairing the visibility of the image displayed at the position where the user tried to bring the pointing object closer. In addition, the user can easily recognize a desired object from a plurality of objects, and can easily select a desired object.

  In addition, the information processing apparatus includes a display distance detection sensor that detects an indication object that has approached the display surface up to the display distance, and an erasure that detects an indication object that has approached the display surface to the erasure determination distance. A determination distance detection sensor, wherein the object display means displays the object on the display device when the display distance detection sensor detects the pointing object, and the approach determination means detects the erasure determination distance detection. When the sensor detects the pointing object, it is determined whether or not the intersection of the straight line connecting the shortest distance between the pointing object and the display surface is included in the area where the object is displayed. The structure which determines may be sufficient.

  According to the above configuration, the pointing object that has approached the display distance is detected by the display distance detection sensor, and the object is displayed at this time. Further, the pointing object approaching the erasure determination distance is detected by the erasure determination distance detection sensor, and at this time, it is determined whether or not the intersection is included in the area where the object is displayed.

  As described above, the display and erasure control of the object can be realized by using the sensor that detects the pointing object that has approached the display distance and the sensor that detects the pointing object that has approached the erasure determination distance.

  The input device is a capacitance type touch panel, and the object display means displays the indication object on the display surface according to a change in capacitance on the display surface detected by the input device. The approach determining means detects that the pointing object has approached the display surface up to the erase determination distance due to a change in capacitance on the display surface detected by the input device. The position of the intersection may be specified based on a change in electrostatic capacitance on the display surface detected by the input device.

  According to the above configuration, since the input device is a capacitance type touch panel, a change in capacitance on the display surface is detected. Therefore, by using this detection result, it can be detected that the pointing object has approached the display surface up to the display distance or the erasure determination distance, and the position of the intersection can be specified.

  In other words, according to the above configuration, a special configuration for detecting that the pointing object has approached the display surface up to the display distance or the erasure determination distance and a special configuration for specifying the position of the intersection are provided. Even without this, it is possible to control the display and deletion of objects.

  The information processing apparatus may be realized by a computer. In this case, a control program for causing the computer to realize the information processing apparatus by causing the computer to operate as each unit of the information processing apparatus, and A computer-readable recording medium on which it is recorded also falls within the scope of the present invention.

  As described above, in the information processing apparatus of the present invention, when the pointing object that is brought close to the display surface so that the user performs an input operation on the display surface approaches the display surface to a predetermined display distance. In addition, an object display means for causing the display device to display an object that can be selected by the user on the display surface, and whether or not the pointing object is moving toward the object displayed by the object display means. The approach display means is configured to end the display of the object when the approach determination means determines that the pointing object has not moved toward the object.

  In addition, as described above, the control method of the information processing apparatus according to the present invention displays a predetermined object displayed on the display surface when the pointing object brought close to the display surface in order to perform an input operation on the display surface. An object display step for causing the display device to display an object that can be selected on the display surface when the user approaches the distance, and the pointing object moves toward the object displayed in the object display step. An approach determining step for determining whether or not the object is displayed, and a display ending step for ending the display of the object when it is determined in the approach determining step that the pointing object has not moved toward the object. It is.

  Therefore, when the pointing object is not close to the display distance, other images displayed on the display surface are not obstructed by the object, and the object is displayed when the user does not select the object after the object is displayed. There is an effect that the user operation is not hindered.

It is a block diagram which shows the principal part structure of the information processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of the data for palette control which the said information processing apparatus uses for the display of a tool palette, etc. FIGS. 4A to 4D are diagrams showing examples of screens displayed on the display / input device by the information processing apparatus. It is a flowchart which shows an example of the process which the said information processing apparatus performs. It is a figure explaining the method to acquire the position of a pointing object and the distance from a pointing object to a display surface with an imaging device. It is a figure explaining the method of detecting that the position of the pointing object, the pointing object has approached the pallet display distance, and the pallet erasing distance by the infrared sensor, and FIG. FIG. 2B shows a method for detecting the pallet display distance, and FIG. 4C shows a method for detecting the pallet erase distance. It is a figure explaining the method to detect the distance to the display surface of a display / input device with the touch pen provided with the infrared rays light emission part and light-receiving part. It is a figure which shows an example of a tool palette. It is a figure which shows the usage example when the screen size of a display / input device is large. It is a figure which shows a prior art and is a figure which shows the example of a screen which displayed the tool palette containing a some icon.

  Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS.

[Configuration of information processing device]
First, the configuration of the information processing apparatus 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram illustrating a main configuration of the information processing apparatus 1. As shown in the figure, a display / input device (display device, input device) 2 and a position / distance sensor (display distance detection sensor, erasure determination distance detection sensor) 3 are connected to the information processing apparatus 1.

  The display / input device 2 is a device that displays an image according to the control of the information processing device 1, receives a user input operation on the display surface, generates an operation signal corresponding to the received input operation, and generates the operation signal It is a device that transmits an operation signal to the information processing device 1. The user can control the operation of the information processing apparatus 1 by performing an input operation on the display surface in accordance with an image displayed on the display surface of the display / input device 2.

  Since the function of displaying an image and the function of detecting a contact position with respect to the display surface are independent, the display / input device 2 can be grasped as two independent devices, a display device and an input device. .

  The position / distance sensor 3 detects the distance between the display surface of the display / input device 2 and the pointing object (a finger or a touch pen of a user who wants to perform an input operation on the display / input device 2), and also displays the input / output device 2 The position of the pointing object on the display surface is detected, and the detection result is transmitted to the information processing apparatus 1. The position and distance detection method will be described later.

  In the illustrated example, the display / input device 2 and the position / distance sensor 3 are separate devices from the information processing device 1, but the information processing device 1 is the display / input device 2 and the position / distance sensor 3. You may provide at least one of these.

  The information processing device 1 is a device that displays an image on the display / input device 2 and that performs processing according to a user input operation received by the display / input device 2, and has the following characteristic features: Execute the process.

  That is, the information processing apparatus 1 displays a tool palette near the display surface of the display / input device 2 when a finger, a touch pen, or the like approaches the display surface. Then, when the finger, the touch pen, or the like gets closer to the display surface, it is confirmed whether the finger, the touch pen, or the like is facing the tool palette.

  Here, when it is confirmed that a finger, a touch pen, or the like is facing the tool palette, the information processing apparatus 1 continues to display the tool palette, and when the tool palette icon is touched, the icon is displayed. Perform the corresponding process.

  On the other hand, when it is confirmed that a finger, a touch pen, or the like is not facing the tool palette, the information processing apparatus 1 ends the display of the tool palette and ends the reception of the input operation using the tool palette.

  Accordingly, the tool palette can be displayed at a position where the user can easily operate and an input operation can be accepted, and the tool palette can be prevented from getting in the way when the user performs an input operation without using the tool palette. The tool palette includes at least one object that can be selected by the user, and each object is associated with a process.

  The information processing apparatus 1 of FIG. 1 includes a display processing unit 10, an input data processing unit 11, a control unit 12, a primary storage unit 13, and a secondary storage unit 14 in order to execute the above processing. Yes.

  The display processing unit 10 has a function as a so-called display driver that performs processing for displaying an image on the display / input device 2. The input data processing unit 11 has a function as a so-called touch panel driver that processes an operation signal received from the display / input device 2.

  The control unit 12 controls the overall operation of the information processing apparatus 1. The control unit 12 controls the palette control unit (object display unit, approach determination unit, process execution unit) 20, position acquisition unit 21, and distance acquisition unit 22. Including.

  The palette control unit 20 performs control related to the tool palette. Specifically, when a finger, a touch pen, or the like approaches the display surface of the display / input device 2, the palette control unit 20 displays a tool palette in the vicinity thereof. In addition, when the finger, the touch pen, or the like further approaches the display surface, the palette control unit 20 checks whether the finger, the touch pen, or the like is facing the tool palette.

  Here, when it is confirmed that the user is heading to the tool palette, the palette control unit 20 continues to display the tool palette, and when a tool palette icon is touched, executes processing corresponding to the icon. . On the other hand, when it is confirmed that it is not directed to the tool palette, the display of the tool palette is terminated, and the acceptance of the input operation by the tool palette is terminated.

  The position acquisition unit 21 receives the detection result of the position / distance sensor 3, generates position information indicating the position of the pointing object with respect to the display surface of the display / input device 2, and sends the generated position information to the pallet control unit 20. Send. More specifically, the position acquisition unit 21 generates, as the position information, coordinates on the display surface that indicate the position of the intersection of the pointing object and the straight display surface that connects the shortest distances between the display object and the display object.

  The distance acquisition unit 22 receives the detection result of the position / distance sensor 3, generates distance information indicating the distance between the pointing object and the display surface of the display / input device 2, and uses the generated distance information as the pallet control unit 20. Send to.

  The primary storage unit 13 is used when the control unit 12 operates, and stores information for maintaining the current input state. In the illustrated example, display data 30 for displaying an image on the display / input device 2 is stored in the primary storage unit 13. The primary storage unit 13 can also be configured by a RAM (Random Access Memory) or the like.

  The secondary storage unit 14 stores various types of data used by the information processing apparatus 1, and any control unit 12 may be used as long as it can store and read data. For example, the secondary storage unit 14 may be configured by a hard disk or the like. . When the control unit 12 is configured by software, software, a driver, and the like for realizing the function of the control unit 12 are also stored in the secondary storage unit 14. In the illustrated example, the secondary storage unit 14 stores pallet control data 31 that the pallet control unit 20 uses to display a tool pallet.

  The palette control data 31 is such that the palette control unit 20 can display the tool palette, and when the icon of the tool palette is selected, the process assigned to the selected icon can be specified. For example, the data shown in FIG. 2 may be used. FIG. 2 is a diagram illustrating an example of the pallet control data 31. The palette control data 31 in FIG. 2 is data in which relative coordinates, icons, and functions are associated with each other.

  The relative coordinates are data for specifying the position where the icon is displayed. As described above, since the tool palette is displayed in the vicinity of the position where the user's finger or the like has approached, in the example of FIG. 2, the display position of the icon is the position generated by the position acquisition unit 21 (the display object and the display object). The position of the point of intersection with the straight display surface connecting the shortest distance to the surface is shown in relative coordinates with the origin (0, 0).

  The icon is displayed at a position indicated by relative coordinates. In the figure, for the sake of explanation, an icon image is shown, but the palette control data 31 only needs to include data that can identify the icon image, and the icon image is included in the palette control data 31. It does not have to be included. For example, an address indicating the storage location of the icon image may be included instead of the icon image.

  The function indicates a function (processing) assigned to the icon. In the illustrated example, functions of a black pen, a red pen, an eraser, a trash can, registration, printing, composition, and help are shown.

  Here, it is assumed that the black pen is a function that displays the handwriting input locus in black, the red pen is a function that displays the handwriting input locus in red, and the eraser is a function that erases the handwriting input locus. doing. In addition, the trash can deletes the specified target, the registration function records the displayed screen, the print function prints the displayed screen, the composition starts text data input function, and the help is It is assumed that this is a function that displays a function guide.

  The palette control data 31 is not limited to the example shown in the drawing as long as it includes data that can identify the function assigned to the icon. Of course, the function assigned to the icon is not limited to these examples.

[Display screen example]
A screen displayed on the display / input device 2 by the information processing apparatus 1 will be described with reference to FIG. FIGS. 3A to 3D are diagrams showing examples of screens displayed on the display / input device 2 by the information processing apparatus 1.

  FIG. 4A shows an example of a screen on which a tool palette is displayed when a finger approaches the display surface of the display / input device 2. As shown in the figure, when the finger approaches the touch panel, the information processing apparatus 1 displays the tool palette 50 at a position away from the position of the approaching finger (indicated by the intersection of broken lines in the figure).

  The illustrated tool palette 50 includes icons a to h. These icons are images and display positions determined based on the palette control data 31. As shown in the figure, by displaying the icons a to h so as to surround the position of the approaching finger, the user can confirm the image of the position where the finger is approached and easily recognize the image of each icon a to h.

  Further, processing (function) is assigned to each of the icons a to h by the palette control data 31, and when the icons a to h are touched, the information processing apparatus 1 performs processing assigned to the icon. Execute.

  FIG. 5B shows an example of a screen when a finger approaches a portion other than the tool palette 50 after the tool palette 50 is displayed. As shown in the figure, the information processing apparatus 1 displays the tool palette 50 when the finger approaches the part other than the tool palette 50, that is, the finger approaches the display surface. When it is not directed to any of them, the display of the tool palette 50 is ended, and the reception of the input operation by the tool palette 50 is also ended.

  FIG. 6C shows an example of a screen when a finger approaches the tool palette 50 after the tool palette 50 is displayed. As shown in the figure, the information processing apparatus 1 displays a tool pallet 50, and when the finger approaches the tool pallet 50, that is, the finger is close to the display surface and the icon a to h is displayed. When the user is heading, the display of the tool palette 50 is continued, and the input operation by the tool palette 50 is also continued. That is, in this case, the user can touch any of the icons a to h to cause the information processing apparatus 1 to execute the process assigned to the icon.

  FIG. 4D shows an example of a screen when the finger is released from the tool palette 50 after the tool palette 50 is displayed. As shown in the drawing, the information processing apparatus 1 ends the display of the tool palette 50 when the finger is removed from the tool palette 50 after the tool palette 50 is displayed, that is, when the finger moves away from the display surface. The acceptance of the input operation using the tool palette 50 is also terminated.

[Process flow]
The display as shown in FIG. 3 is realized by the following processing, for example. Here, the flow of processing executed by the information processing apparatus 1 will be described with reference to FIG. FIG. 4 is a flowchart illustrating an example of processing executed by the information processing apparatus 1.

  First, the pallet control unit 20 instructs the distance acquisition unit 22 to acquire distance information from the position / distance sensor 3 (S1). Then, the palette control unit 20 refers to the acquired distance information and confirms whether there is anything approaching the display surface of the display / input device 2 up to the palette display distance (display distance) (S2). If there is no approaching object (NO in S2), the process returns to S1.

  The palette display distance is a distance that is determined in advance as a threshold for determining whether or not to display the tool palette. The palette display distance may be appropriately set according to the screen size of the display / input device 2 or the like.

  On the other hand, when it is confirmed that there is something approaching the pallet display distance (YES in S2), the pallet control unit 20 instructs the position acquisition unit 21 to acquire position information from the position / distance sensor 3. (S3).

  Subsequently, the palette control unit 20 refers to the position information acquired by the position acquisition unit 21, identifies the position where the user's finger approaches, and displays a tool palette around the identified position (S4).

  Specifically, the palette control unit 20 reads the image of each icon (object) and the relative coordinates of the icon from the palette control data 31 stored in the secondary storage unit 14. Then, the display position of each icon is determined according to the read relative coordinates with the coordinates indicated by the position information acquired by the position acquisition unit 21 as the center, and the display processing unit 10 is instructed to display each icon at the determined display position. Display. As a result, the tool palette is displayed so as to surround the position where the finger approaches as shown in FIG.

  Next, the pallet control unit 20 instructs the distance acquisition unit 22 to acquire distance information from the position / distance sensor 3 (S5). Then, referring to the acquired distance information, it is confirmed whether or not the user's finger has approached the pallet erasure distance (erase determination distance) (S6). Specifically, the pallet control unit 20 confirms whether the distance indicated by the acquired distance information is equal to or less than the pallet erasure distance.

  The palette erase distance is a distance used for determining whether or not the user's finger is facing the tool palette, and is set to a value shorter than the palette display distance. The palette deletion distance may be set as appropriate according to the palette display distance.

  If it is confirmed that the user's finger is close to the pallet erase distance (YES in S6), the pallet control unit 20 proceeds to the process of S7. On the other hand, when it is confirmed that the user's finger has not approached the pallet erase distance (NO in S6), the pallet control unit 20 confirms whether or not the user's finger is separated from the pallet display distance (S8). ). Specifically, the palette control unit 20 checks whether or not the distance indicated by the distance information acquired in S5 is larger than the palette display distance.

  In S8, when it is confirmed that the user's finger is not separated from the pallet display distance (NO in S8), the pallet control unit 20 returns to the process of S5 and acquires the distance information again. On the other hand, when it is confirmed that the user's finger is separated from the palette display distance (YES in S8), the palette control unit 20 instructs the display processing unit 10 to delete the tool palette (S9), and S1 Return to the process.

  In S <b> 7, the pallet control unit 20 instructs the position acquisition unit 21 to acquire position information from the position / distance sensor 3. Then, the palette controller 20 checks whether or not the user's finger is approaching the tool palette, in other words, whether or not the user's finger is facing the tool palette (S10).

  Specifically, the palette control unit 20 specifies the position of the user's finger from the position information acquired by the position acquisition unit 21. Then, when the coordinates of the specified finger position are included in the area where the tool palette (icon) is displayed on the display surface, it is determined that the user's finger is approaching the tool palette.

  In S10, when it is confirmed that the user's finger is approaching the tool palette (YES in S10), the palette control unit 20 proceeds to the process of S11. On the other hand, when it is confirmed that the user's finger is approaching a part other than the tool palette (NO in S10), the palette control unit 20 instructs the display processing unit 10 to delete the tool palette (S12). ).

  Then, the palette control unit 20 confirms the presence or absence of a touch operation (S13). The presence / absence of the touch operation is confirmed based on whether or not the display / input device 2 has received the operation signal indicating that the touch operation has been received from the input data processing unit 11.

  Here, when it is confirmed that the touch operation has been performed (YES in S13), the pallet control unit 20 executes a process according to the touch position (S14). For example, if a process is assigned to the touched position, the process is executed. For example, when it is in the state which accepts handwriting input, a line segment is displayed along the locus | trajectory of handwriting input. Of course, if no process is assigned to the touched position, the process is not performed. It should be noted that the processing when touching other than the tool palette may be performed by a configuration other than the palette control unit 20.

  On the other hand, when it is not confirmed that the touch operation has been performed (NO in S13), the pallet control unit 20 instructs the distance acquisition unit 22 to acquire the distance information from the position / distance sensor 3 (S15). ), It is confirmed whether or not the user's finger is separated from the palette display distance (S16). This confirmation is performed in the same manner as in S8. When it is confirmed that the distance is not longer than the palette display distance (NO in S16), the process returns to S13, and when it is confirmed that the distance is longer than the palette display distance (S16). If YES, the process returns to S1.

  In S11, the palette control unit 20 confirms whether or not a touch operation has been performed on the tool palette being displayed. When it is confirmed that the touch operation has been performed (YES in S11), the palette control unit 20 instructs the display processing unit 10 to delete the tool palette (S17), and the palette corresponding to the touch position. The function is executed (S18).

  Specifically, the pallet control unit 20 determines whether the coordinate of the position where the touch operation is performed indicated by the operation signal received from the input data processing unit 11 is included in which area of the displayed icon. It is determined which palette function has been selected. Then, the pallet control unit 20 refers to the pallet control data 31 and specifies and executes a process corresponding to the selected pallet function. Thereafter, the process returns to S1.

  On the other hand, when the touch operation is not confirmed in S11 (NO in S11), the palette control unit 20 instructs the distance acquisition unit 22 to acquire the distance information from the position / distance sensor 3 (S19), and the user It is confirmed whether or not the finger is further than the pallet display distance (S20). This confirmation is performed in the same manner as S8 and S16.

  If it is confirmed that the distance is not longer than the pallet display distance (NO in S20), the process returns to S11. If it is confirmed that the distance is longer than the pallet display distance (YES in S20), the display processing unit. 10 is instructed to delete the tool palette (S21), and the process returns to S1.

[Position / distance detection method]
Next, a method for detecting the position of the pointing object (such as a user's finger) on the display surface of the display / input device 2 and the distance from the display surface will be described.

  First, a method for acquiring an image near the display surface of the display / input device 2 and acquiring the position and distance by analyzing the acquired image will be described with reference to FIG. FIG. 5 is a diagram for explaining a method of acquiring the position of the pointing object and the distance from the pointing object to the display surface using the imaging apparatus.

  As illustrated, imaging devices 5 are provided at the four corners of the display / input device 2. That is, in the illustrated example, it is assumed that the four imaging devices 5 function as the position / distance sensor 3.

  Each imaging device 5 can capture an image of the pointing object (such as a user's finger) that has approached the display distance of the display / input device 2 to the palette display distance, and can also capture the pointing object that has approached the palette deletion distance. Is arranged. In addition, an image captured by each imaging device 5 is transmitted to the position acquisition unit 21 and the distance acquisition unit 22.

  As described above, by analyzing the images obtained by capturing the pointing object from different positions, the position acquisition unit 21 can specify the position of the pointing object with respect to the display surface. Specifically, it is possible to specify the coordinates on the display surface indicating the position of the intersection point of the straight display surface connecting the shortest distance between the pointing object and the display surface. The distance acquisition unit 22 can specify the distance from the pointing object to the display surface.

  Here, the information processing apparatus 1 only needs to be able to specify that the pointing object has approached the pallet display distance and the pallet deletion distance, so that the pointing object can be measured without measuring the distance from the pointing object to the display surface. It may be detected that is approaching the palette display distance and approaching the palette deletion distance.

  A method of detecting that the pointing object has approached the pallet display distance and the pallet deletion distance will be described with reference to FIG. FIG. 6 is a diagram for explaining a method for detecting the position of the pointing object, the approaching of the pointing object to the pallet display distance, and the pallet erasing distance by the infrared sensor, and FIG. FIG. 4B shows a method for detecting the pallet display distance, and FIG. 4C shows a method for detecting the pallet erase distance.

  As shown in FIG. 5A, here, a plurality of light emitting portions 7 are provided along one side on the left side of the display surface of the display / input device 2, and the side facing the one side is provided. A light receiving portion 8 for receiving the light emitted from each light emitting portion 7 is provided. Similarly, a plurality of light emitting units 7 are provided along one side on the upper side of the display surface, and a light receiving unit 8 that receives light emitted from each light emitting unit 7 is provided along a side opposite to the one side. It has been.

  According to the illustrated configuration, when the pointing object approaches the display surface of the display / input device 2, the light emitted from the light emitting unit 7 is blocked by the pointing object and is not received by the light receiving unit 8. Accordingly, the display surface indicating the position of the pointing object on the display surface, specifically the position of the intersection of the straight display surface connecting the shortest distance between the pointing object and the display surface, from the position of the light receiving unit 8 that could not receive light. The upper coordinates can be specified.

  By providing such a combination of the light emitting unit 7 and the light receiving unit 8 in two stages, it is possible to detect that the pointing object has approached the pallet display distance and the pallet erase distance. Further, by specifying the light receiving unit 8 that cannot receive light, it is possible to specify the position where the pointing object is approaching the display surface.

  That is, as shown in FIG. 5B, the light emitting unit 7 is arranged so that light passes through the position separated from the display surface by the pallet display distance in parallel with the display surface, and the light emitted by the light emitting unit 7 can be received. The light receiving unit 8 is disposed at the position. As a result, when the distance from the pointing object (the user's finger in the illustrated example) to the display surface becomes equal to or smaller than the pallet display distance, the light receiving unit 8 cannot receive light. The approach can be detected.

  Further, as shown in FIG. 5C, the light emitting unit 7 is arranged so that light passes through a position away from the display surface by a pallet erasing distance in parallel with the display surface, and light emitted from the light emitting unit 7 can be received. The light receiving unit 8 is disposed at the position. As a result, when the distance from the pointing object (the user's finger in the illustrated example) to the display surface becomes equal to or smaller than the pallet erasing distance, the light receiving unit 8 cannot receive light. The approach can be detected.

  That is, in this case, the position acquisition unit 21 is arranged horizontally with respect to the position of the light receiving unit 8 that could not receive light among the light receiving units 8 arranged vertically with respect to the display surface. The coordinates on the display surface that indicate the position of the intersection between the pointing object and the straight display surface connecting the shortest distance between the display object and the position of the light receiving unit 8 that has not received light are specified. . Then, the specified coordinates are transmitted to the pallet control unit 20 as position information.

  Further, the distance acquisition unit 22 transmits information indicating whether or not there is a light receiving unit 8 arranged at the pallet erasing distance to the pallet control unit 20 as distance information. Similarly, the distance acquisition unit 22 transmits information indicating whether or not there is a light receiving unit 8 disposed at the pallet erasing distance to the pallet control unit 20 as distance information.

  The distance acquisition unit 22 notifies the pallet control unit 20 when the light receiving unit 8 arranged at the pallet display distance that could not receive light is confirmed, and the light receiving unit arranged at the pallet erasing distance. When the part 8 that could not receive light is confirmed, the pallet control part 20 may be notified to that effect.

  The display / input device 2 may be one that detects a user's touch operation by providing a light emitting unit and a light receiving unit. In this case, the light emitting unit and the light receiving unit are provided in three stages. It will be. Further, instead of the infrared light emitting unit 7 and the light receiving unit 8, a microwave transmitting unit and a receiving unit may be arranged to detect the approach of the pointing object.

  Further, in the above description, an example in which the position of the pointing object and the distance from the pointing object to the display surface are detected by one position / distance sensor 3 has been described. However, the position of the pointing object and the distance from the pointing object to the display surface are illustrated. The distance may be detected by a separate sensor.

  For example, a touch pen for performing a touch operation on the position / distance sensor 3 may have a distance measuring function. This will be described with reference to FIG. FIG. 7 is a diagram for explaining a method of detecting the distance to the display surface of the display / input device 2 with the touch pen 6 including the infrared light emitting unit 6a and the light receiving unit 6b.

  As shown in the figure, the light emitted from the light emitting unit 6 a provided in the touch pen 6 is reflected by the display surface of the display / input device 2, and the reflected light is received by the light receiving unit 6 b provided in the touch pen 6. The distance from 6 to the display surface can be measured. The measured distance is transmitted from the touch pen 6 to the information processing apparatus 1 by wireless or wired communication.

  The position acquisition method is not particularly limited as long as it can acquire the position of the pointing object (such as a user's finger) on the display surface, and the distance acquisition method can also acquire the distance from the display surface. If it does not specifically limit, methods other than illustrated can also be applied.

  For example, when the display / input device 2 is a capacitance type touch panel, a change in the capacitance with respect to the display surface can be detected. Therefore, the approach of the finger is detected from the detection result, and the position where the finger approaches In addition, the distance between the finger and the display surface may be acquired.

  That is, the capacitive touch panel forms a uniform electric field on the display surface. When a conductor such as a user's finger approaches the electric field, the capacitance on the display surface changes. Therefore, the capacitive touch panel detects this change, and the user can operate the display surface. Is detected.

  For this reason, the position acquisition unit 21 receives information indicating the position where the capacitance on the display surface has changed from the capacitance type touch panel, and determines the coordinates of the position as the shortest distance between the pointing object and the display surface. It can be specified as coordinates on the display surface indicating the position of the intersection with the display surface of the straight line to be connected.

  Further, the distance acquisition unit 22 receives information indicating the capacitance on the display surface from the capacitance type touch panel, and receives the received capacitance and a predetermined threshold (a threshold for palette display distance and a palette erase distance). It is also possible to determine whether the pointing object is close to the pallet display distance or the pallet erasure distance by comparing with the threshold value), and transmit the determination result to the pallet control unit 20 as distance information.

[Modification of tool palette]
In the above description, an example (FIG. 3) is shown in which the tool palette 50 in which icons are arranged in a circle around the approaching position of the pointing object is shown. However, the tool palette can be selected by touching by the user, It suffices to include at least one object to which some kind of processing is assigned, and is not limited to the example of FIG.

  For example, a tool palette as shown in FIG. 8 may be displayed. FIG. 8 is a diagram illustrating an example of a tool palette. In the illustrated example, a tool palette 50 'in which six icons are arranged in a rectangular shape is displayed on the side of the finger close to the display surface. There are six icons a, b, c, f, g, and h included in the tool palette 50 '.

  Thus, the configuration and display position of the tool pallet are not particularly limited, but the position on which the pointing object will come into contact (for example, the position on the display surface corresponding to the position of the pointing object when approaching the pallet erase distance) ) Preferably does not display the tool palette. In other words, it is preferable to display the tool palette so that the icon included in the tool palette is at a predetermined distance from the intersection of the straight line connecting the shortest distance between the pointing object and the display surface. .

  This prevents the user from accidentally touching the icon in the tool palette when the user tries to perform a touch operation on the display surface with the intention of operating another object that is already displayed. Can do.

[Display / input device]
The display / input device 2 connected to the information processing device 1 displays an image according to the control of the information processing device 1, detects a user's touch operation, and transmits an operation signal indicating the result to the information processing device 1. If it is a thing, it will not specifically limit. However, when the screen size of the display / input device 2 is large, the effect of applying the information processing device 1 is particularly great.

  This will be described with reference to FIG. FIG. 9 is a diagram showing an example of use when the screen size of the display / input device 2 is large. As shown in the figure, when the screen size of the display / input device 2 is large, it can also be used as a device for displaying materials and the like presented to the participants by the presenter in meetings, presentations, lectures, and the like.

  According to the information processing apparatus 1, the tool palette is displayed in the vicinity of the approached position by bringing a finger, a touch pen, or the like closer to the display / input device 2, so that the user can start from any position. Input operations using the tool palette can be easily performed.

  In addition, when the screen size of the display / input device 2 is large as described above, various icons can be displayed in addition to the tool palette. When used in meetings, presentations, lectures, etc., It is considered that handwriting input is often performed. Therefore, it is considered that the user often brings a finger or the like close to the display surface without intending to perform an operation using the tool palette.

  According to the information processing apparatus 1, since the tool palette is deleted when the user's finger or the like is not facing the tool palette, the tool palette does not interfere with the user's operation and is very convenient.

[Modification]
In the above, when the pointing object such as the user's finger approaches the pallet deletion distance, the pointing object moves toward the tool pallet by determining whether or not the position of the pointing object is on the tool pallet. An example of determining whether or not there has been described.

  However, the method for determining whether or not the pointing object is moving toward the tool palette is not limited to the above example. For example, even if the pointing object is imaged a plurality of times at different timings and it is determined whether or not the pointing object is moving toward the tool palette based on the position change of the pointing object specified from the captured images Good. Further, for example, it may be determined whether or not the pointing object is moving toward the tool palette by analyzing a moving image obtained by capturing the pointing object.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  Finally, each block of the information processing apparatus 1, particularly the control unit 12, may be realized by hardware by a logic circuit formed on an integrated circuit (IC chip), or a CPU (Central Processing Unit) is used. It may be realized by software.

  In the latter case, the information processing apparatus 1 stores a CPU that executes instructions of a program that realizes each function, a ROM (Read Only Memory) that stores the program, a RAM that expands the program, the program, and various data. A storage device (recording medium) such as a memory is provided. An object of the present invention is to provide a recording medium in which a program code (execution format program, intermediate code program, source program) of a control program of the information processing apparatus 1 which is software for realizing the above-described functions is recorded so as to be readable by a computer. This can also be achieved by supplying the information processing apparatus 1 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and disks including optical disks such as CD-ROM / MO / MD / DVD / CD-R. IC cards (including memory cards) / optical cards, semiconductor memories such as mask ROM / EPROM / EEPROM (registered trademark) / flash ROM, PLD (Programmable logic device) and FPGA (Field Programmable Gate) Logic circuits such as (Array) can be used.

  The information processing apparatus 1 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited as long as it can transmit the program code. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication network, and the like can be used. The transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, even with wired lines such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, and ADSL (Asymmetric Digital Subscriber Line) line, infrared rays such as IrDA and remote control, Bluetooth (registered trademark), IEEE 802.11 wireless, HDR ( It can also be used by wireless such as High Data Rate (NFC), Near Field Communication (NFC), Digital Living Network Alliance (DLNA) (registered trademark), mobile phone network, satellite line, and digital terrestrial network.

  The present invention is suitable for an information processing apparatus that is used by being connected to a display / input device having a function of displaying an image on a display surface and a function of receiving an input operation on the display surface, and an information processing device including the display / input device. Can be applied to.

1 Information processing device 2 Display / input device (display device, input device)
3 Position / distance sensor (display distance detection sensor, erase determination distance detection sensor)
20 Pallet control unit (object display means, approach determination means, process execution means)

Claims (8)

An information processing apparatus that displays an image on a display device and operates according to an operation signal received from an input device that detects an input operation performed on a display surface of the display device,
The user selects on the display surface when the pointing object that has been brought close to the display surface to perform an input operation on the display surface approaches a predetermined display distance with respect to the display surface. An object display means for displaying an object that can be displayed on the display device;
The input device is a capacitive touch panel,
The object display means is
It is determined whether or not the pointing object has approached the display distance by comparing the capacitance on the display surface detected by the input device with a predetermined threshold for display distance, and the input device Determining whether or not the pointing object has approached an erasure determination distance shorter than the display distance by comparing the capacitance on the display surface detected by the predetermined erasure determination distance threshold,
The position where the capacitance on the display surface detected by the input device is changed is not included in the area where the object is displayed when it is determined that the pointing object has approached the erase determination distance In this case, the information processing apparatus ends the display of the object.   The information processing apparatus according to claim 1, further comprising: a process execution unit that executes a process previously assigned to the object when the object is selected by the pointing object.   The object display means is predetermined from an intersection of a straight line connecting a shortest distance between the pointing object and the display surface when the pointing object approaches the display surface up to the display distance. The information processing apparatus according to claim 1, wherein the object is displayed at a position separated by a distance.   The information processing apparatus according to claim 3, wherein the object display unit displays a plurality of the objects so as to surround the intersection. In a state that accepts handwritten input, when an area on the touch operation other than the region that is displayed in the upper Symbol object is performed, and further comprising means for displaying a line segment along a trajectory of handwriting The information processing apparatus according to any one of claims 1 to 4. A control method for an information processing apparatus that displays an image on a display device and that operates according to a detection result of an input operation performed on a display surface of the display device detected by an input device including a capacitive touch panel. And
The display in which the input device has detected whether or not the pointing object that has been approached to the display surface for the user to perform an input operation on the display surface has approached the display surface to a predetermined display distance. A first determination step of determining by comparing the capacitance on the surface with a predetermined display distance threshold;
An object display step for causing the display device to display an object that the user can select on the display surface when it is determined in the first determination step that the pointing object has approached the display distance ;
By comparing the capacitance on the display surface detected by the input device with a predetermined threshold value for erasure determination distance, it is determined whether or not the pointing object has approached an erasure determination distance shorter than the display distance. A second determining step,
When the second determination step determines that the pointing object has approached the erasure determination distance, the position where the capacitance on the display surface detected by the input device has changed is displayed on the object. And a display end step of ending display of the object when the object is not included in the area . A control program for operating the information processing apparatus according to any one of claims 1 to 5 , wherein the control program causes a computer to function as each of the means. A computer-readable recording medium on which the control program according to claim 7 is recorded.

Images