JP2010182135A - Input device and input method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an input device and an input method, allowing correct extraction of subsequent contact coordinates even when Y coordinates or the like overlap each other by movement of respective points, in a touch panel. <P>SOLUTION: In this input device specifying a contact position of a hand of a user from a detection position on an operation face of the touch panel and performing input operation, coordinates on the operation face are represented by an X coordinate axis and a Y coordinate axis orthogonal to each other. When the coordinate detected on the Y coordinate axis is smaller than a coordinate number detected on the X coordinate axis with a direction corresponding to a long direction of the hand of the user as the Y coordinate axis, the coordinate number detected on the X coordinate axis is set as the number of contact points of the hand of the user. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an input device and an input method for a user to input a command or the like to a device, and more specifically, using a body part such as a hand based on information displayed on a display or the like by a user. The present invention relates to an input device and an input method capable of inputting commands and the like.

  With the recent increase in functionality of information equipment terminals, user interfaces that are easier for users to use are required. An example of an easy-to-use user interface is a touch panel display. The touch panel display is a device in which a user inputs a command or the like to an information device terminal or the like by pressing a GUI (Graphical User Interface) part displayed on the display. From this, it can be said that the touch panel display is an interface that even a beginner can operate intuitively.

  However, in a touch panel display, since a device for detecting a touch (hereinafter referred to as an operation unit) and a display unit are integrated, there is a problem that the touch panel display cannot be operated unless the user has it.

In order to solve this problem, the technique of Patent Document 1 separates an operation unit and a display unit, detects an operator's hand placed on the operation unit, and forms a computer graphic (hereinafter referred to as CG) hand shape. A model (hereinafter simply referred to as a hand model) is displayed superimposed on a GUI displayed on the display unit. Thus, even when the user is away from the display unit, the user can perform an operation while maintaining an intuitive operational feeling of the touch panel display.
JP 2006-72854 A

  However, according to the conventional technique described above, the correspondence between the contact coordinates acquired in the initial stage and the coordinates of the contact coordinates moved during the operation cannot be obtained depending on the type of the touch panel that detects the contact coordinates of a plurality of fingers. Exists.

  For example, as a touch panel detection method, there is an optical method in which coordinates are specified by blocking light such as infrared rays with a finger. This optical type is generally used in many ATMs. When contact of a plurality of fingers is detected by such an optical method, more coordinate candidates than the number of touched fingers are necessarily extracted. An example at this time is shown in FIG.

  FIG. 19 shows another point C (x1, y1) and point D (x2, y2) on the touch panel even though the point A (x1, y2) and the point B (x2, y1) are actually in contact with each other. ) Is detected. In this case, since the X coordinate and the Y coordinate are detected independently, intersections are extracted by the number of combinations of the respective coordinates.

  In the case of such a method, if the sampling rate at the time of coordinate detection is sufficient (about 20 Hz or more), the user will consciously detect each because there is a time difference unless two or more points are consciously touched simultaneously. Can do. For example, suppose that point A first touches and then point B touches. Here, first, the detected coordinates (x1, y2) are stored as the coordinates of the point A. After that, at the next timing, the points B (x2, y1), C (x1, y1), and D (x2, y2), which are three new coordinates, are detected. It can be estimated that the points except (x1, y2) should be newly detected by the touch of the user's finger. Thereby, (x2, y1) is stored as the coordinates of the point B where the user touches. By performing such processing, two or more coordinates can be easily detected.

  However, even if each point can be detected in the initial stage as shown in FIG. 19, if each point moves and the Y coordinates and the like overlap each other, there are cases where subsequent contact coordinates cannot be extracted correctly. This case will be described with reference to FIG.

  FIG. 20 shows each state of the contact location (shaded portion) detected by the touch panel, the point where the operator actually touched the touch panel (point A and point B), and the extraction result of each coordinate at that time. . In the state a, it is assumed that the contact coordinates of each point (point A, point B) can be detected by the above method. Next, when the point moves from the state a to the state b, in the state b, there is a problem that it becomes difficult to know which point the Y coordinate has moved.

  The present invention solves the above-described conventional problems, and provides an input device and an input method capable of correctly extracting subsequent contact coordinates even when Y coordinates and the like overlap each other on a touch panel. With the goal.

  In order to solve the above-described conventional problems, a first aspect of the present invention is directed to an input device. The present invention is an input device that performs an input operation by specifying a contact position of a user's hand from a detection position on an operation surface of a touch panel, and coordinates on the operation surface are represented by an X coordinate axis and a Y coordinate axis that are orthogonal to each other, Assuming that the direction corresponding to the longitudinal direction of the user's hand is the Y coordinate axis, when the number of coordinates detected on the Y coordinate axis is less than the number of coordinates detected on the X coordinate axis, the number of coordinates detected on the X coordinate axis is determined by the user. The number of contact points of the hand.

  Further, when the number of coordinates detected on the Y coordinate axis is one less than the number of coordinates detected on the X coordinate axis, when a new coordinate is detected on the Y coordinate axis, the distance of the Y coordinate of the detection position before and after the detection. Based on the above, the two detection positions where the Y coordinate is overlapped are specified, and the detection position with the larger change in the X coordinate is specified as the contact position of the user's hand among the two detection positions before and after the detection.

  The second aspect of the present invention is directed to an input method. The present invention is an input method for performing an input operation by specifying a contact position of a user's hand from a detection position on an operation surface of a touch panel, wherein coordinates on the operation surface are represented by an X coordinate axis and a Y coordinate axis that are orthogonal to each other, Assuming that the direction corresponding to the longitudinal direction of the user's hand is the Y coordinate axis, and the number of coordinates detected on the Y coordinate axis is smaller than the number of coordinates detected on the X coordinate axis, the number of coordinates detected on the X coordinate axis is A step of determining the number of contact points of the user's hand.

  Further, when the number of coordinates detected on the Y coordinate axis is one less than the number of coordinates detected on the X coordinate axis, when a new coordinate is detected on the Y coordinate axis, the Y coordinate of the detection position before and after the detection is detected. Based on the distance, the step of identifying the two detection positions where the Y coordinate overlapped, and the detection position with the larger change in the X coordinate of the two detection positions before and after the detection as the contact position of the user's hand Further comprising the step of identifying.

  According to the input device and the input method of the present invention, even when each point moves and the Y coordinate and the like overlap each other on the touch panel, the subsequent contact coordinates can be correctly extracted. Therefore, when a hand model is created based on the extracted contact coordinates and displayed on the display unit, the user's hand shape can be accurately reflected in the hand model. As a result, the user can perform an operation without feeling uncomfortable.

  FIG. 1 is a diagram for explaining the outline of the operation of the input device according to the embodiment of the present invention. Hereinafter, the operation of the input device will be briefly described with reference to FIG.

  As shown in FIG. 1, the input device includes a contact coordinate detection unit 1 and a calculation unit 2. The contact coordinate detection unit 1 detects an area touched by the user's hand (hereinafter referred to as an operator) placed on the operation surface of the contact coordinate detection unit 1, and detects information on the detected contact area (hereinafter simply referred to as contact). (Referred to as region information) is output to the calculation unit 2. The contact coordinate detection unit 1 is, for example, a touch panel that detects contact at multiple points. Note that the contact coordinate detection unit 1 only needs to be able to detect contact at a plurality of points, and may be a general device using an optical method, a capacitance method, a pressure-sensitive method, a resistance film method, or the like.

  The calculation unit 2 creates display data using the contact coordinates detected by the contact coordinate detection unit 1 and outputs the display data to the display unit 3. Specifically, the calculation unit 2 creates a CG hand model using the contact coordinates, creates display data in which the hand model is superimposed on the GUI, and outputs the display data to the display unit 3. Further, the calculation unit 2 determines a command intended by the operator using the positional relationship between the created hand model and the GUI part, and outputs the determined command to the operation target device 4 such as an information terminal device. Note that the arithmetic unit 2 may be a general personal computer (hereinafter referred to as a PC), or may be a general-purpose dedicated machine incorporating a dedicated graphic IC or the like. The calculation unit 2 includes an interface that receives a signal input from the contact coordinate detection unit 1, a CPU, a ROM, and a RAM, a bus that connects the modules, and an interface that outputs a calculation result as an image.

  The display unit 3 performs display using the display data input from the calculation unit 2. The display unit 3 is a display such as a liquid crystal display or a CRT, for example. The operation target device 4 executes the command input from the calculation unit 2.

  With the above configuration, the operator operates the hand model by moving the hand on the operation surface of the contact coordinate detection unit 1 while visually observing the hand model superimposed on the GUI part displayed on the display unit 3. Thus, the operation target device 4 can be operated.

  In the present embodiment, as an example of the installation position of the contact coordinate detection unit 1, a configuration as shown in FIG. 2 is assumed as the installation position on the car. FIG. 2A is a diagram illustrating an example when the vehicle is attached to the center console of the vehicle. FIG. 2B is a diagram illustrating an example when the contact coordinate detection unit 1 is attached to the handle of the vehicle. FIG.2 (c) is a figure which shows an example when the contact coordinate detection part 1 is attached to the operation part of the door of a vehicle. By attaching the contact coordinate detection unit 1 to the position of the vehicle as shown in FIG. 2A, it is known in advance that the operator who is a driver places his left hand in the direction of the arrow with respect to the contact coordinate detection unit 1. .

  Here, the direction of the hand means a direction (longitudinal direction of the hand) from the base point indicating the base of the hand toward the fingertip position point of the middle finger when described with reference to FIG. As shown in FIG. 6A, when the coordinates on the operation surface of the contact coordinate detection unit 1 are expressed by the X coordinate axis and the Y coordinate axis orthogonal to each other, the direction corresponding to the longitudinal direction of the user's hand is the direction of the Y coordinate axis. .

  In addition, by attaching to the position of the vehicle as shown in FIG. 2 (b), it is known in advance that the operator who is a driver places the left hand or the right hand in the direction of the arrow with respect to the contact coordinate detection unit 1. Further, by attaching to the vehicle position as shown in FIG. 2C, it is known in advance that the operator who is a driver places his right hand in the direction of the arrow with respect to the contact coordinate detection unit 1.

  Hereinafter, the configuration and operation of the input device will be described in detail. FIG. 3 is a diagram illustrating functional blocks of the input device 100. As shown in FIG. 3, the input device 100 includes a contact coordinate detection unit 1 and a calculation unit 2. The calculation unit 2 includes a contact coordinate calculation unit 201, a related ID creation unit 202, a movement amount calculation unit 203, a finger bending angle calculation unit 204, a model shape determination unit 205, a coordinate history storage unit 206, and a model holding unit. Unit 207, GUI parts holding unit 208, superimposed image creation unit 209, collision determination unit 210, and command transmission unit 211.

  FIG. 4 is a flowchart for explaining the operation of the input device. When the start switch is turned on by an operator or the like, the input device starts operation.

  First, in step S <b> 101, the contact coordinate detection unit 1 determines whether or not the operator's hand is placed on the operation surface of the contact coordinate detection unit 1. When the operator's hand is not placed, the contact coordinate detection unit 1 enters a process waiting state, and when the operator's hand is placed, the process proceeds to step S102. Note that the determination in step S101 can be made based on whether or not contact is detected.

  Next, in step S102, the contact coordinate calculation unit 201 uses the independent values of the two axes (X and Y coordinates) orthogonal to each other output from the contact coordinate detection unit 1 to determine the contact operator. A coordinate value indicating the positions of the four fingertips (hereinafter referred to as fingertip position points) is calculated. A method for calculating coordinates will be described in detail later.

  Next, in step S <b> 103, the related ID creation unit 202 assigns IDs to the coordinate values calculated by the contact coordinate calculation unit 1, and stores them in the coordinate history storage unit 206. The ID assigning method will be described in detail later.

  Next, in step S104, the movement amount calculation unit 203 determines whether or not four fingertip position points have been acquired. If acquired, the process proceeds to step S105. If not acquired, the process returns to step S102.

  Next, in step S <b> 105, the movement amount calculation unit 203 determines whether or not an initial position point that is a fingertip position point in a state where the operator has opened his hand (a state where the finger is extended) has been acquired. If the initial position point has been acquired, the process proceeds to step S107. If the initial position point has not been acquired, the process proceeds to step S106. Here, the determination of whether or not the initial position point has been acquired is to determine whether or not the initial position point has been acquired after the input device has been activated this time.

  In step S <b> 106, the movement amount calculation unit 203 notifies the operator of a message that prompts the user to open the hand placed on the contact coordinate detection unit 1 and acquires the initial position point. This message is displayed on the display unit 3 and notified to the operator, for example. Thereafter, the movement amount calculation unit 203 uses the acquired initial position point to create a hand model in which the operator has opened his hand (hereinafter referred to as an initial hand model). The initial position point and initial hand model data acquired in step S106 are stored in the model holding unit 207. Thereafter, the process returns to step S102. That is, the input device 100 acquires the initial position point and the initial hand model when the operator places his hand on the contact coordinate detection unit 1 for the first time after activation. The method for acquiring the initial position point and the initial hand model will be described in detail later.

  When the operator changes or when the left and right hands of the operator are switched, the initial position point and the initial hand model that have already been acquired are discarded, and the operation of step S106 is performed again to perform the initial position point. And the initial hand model may be updated. In addition, the initial position point and the initial hand model may be newly acquired by performing the operation of step S106 in response to the operator's instruction without acquiring the initial position point and the initial hand model every time the computer is activated. If the contact coordinate detection unit 1 does not detect a hand contact for a predetermined period in step S101, the already acquired initial position point and initial hand model are discarded, and the operation in step S106 is performed again to obtain a new initial position. Point and initial hand models may be obtained.

  In step S107, the movement amount calculation unit 203 calculates the movement amount of the fingertip position point from the initial position point.

  Next, in step S108, the finger bending angle calculation unit 204 determines the bending angle of the finger of the hand model based on the amount of movement of the fingertip position point from the initial position point acquired in step S107. The method for determining the finger bending angle will be described in detail later.

  Next, in step S109, the model shape determining unit 205 reads the hand model held in the model holding unit 207 using the bending angle of each finger joint of the hand model determined in S108 and reflects the deformation. Here, when the initial hand model is held in the model holding unit 207, the initial hand model is read to reflect the deformation.

  Note that the hand model held in the model holding unit 207 is preferably a CG (Computer Graphics) model having a general skeleton (hereinafter referred to as a bone structure). An example of a hand model held by the model holding unit 207 is shown in FIG. The hand model shown in FIG. 5 includes a plurality of polygons and lines indicating bone structures. The file format for holding the hand model in the model holding unit 207 may be a general format. As an example, there is a file format that stores the vertex coordinates of a polygon. The hand model may also hold texture information and amplify the realism of the hand model.

  Next, in step S110, the superimposed image creation unit 209 reads the GUI parts previously held by the GUI parts holding unit 208, and creates an operation image in which the read GUI parts are arranged. Here, the GUI part is a button or the like to which a control command displayed on the display unit 3 is assigned. The GUI parts and the arrangement method thereof may be general. After that, the superimposed image creation unit 209 creates a superimposed image by superimposing the hand model reflected in the deformation in step S109 and the operation image on which the GUI parts are arranged, and displays the superimposed image on the display unit.

  Next, in step S111, the collision determination unit 210 determines whether or not there is a collision between the GUI part and the fingertip position point of the hand model. Details of the collision determination will be described later. If it is determined that there is no collision, the processes in steps S101 to S111 are repeated, and the hand model in the superimposed image is deformed according to the movement of the operator's hand. If it is determined that there is a collision, the collision determination unit 210 notifies the command transmission unit 211 of the command assigned to the GUI part with the collision, and the process proceeds to step S112.

  In step S112, the command transmission unit 211 transmits the notified command to the operation target device 4, and the process returns to step S101. The operation target device 4 receives and executes the transmitted command. By repeating the above processing, the input device 100 operates the operation target device 4 by repeatedly transmitting a command. Note that the processing ends when the start switch is turned off by an operator or the like.

  By the operation described above, the operator can operate the hand model in the superimposed image by moving his / her hand while viewing the superimposed image displayed on the display unit. Then, the operator can operate the operation target device 4 by operating the hand model and superimposing the position of the fingertip on the GUI part.

  Below, each step demonstrated using FIG. 4 is demonstrated in detail.

  FIG. 6 is a diagram for explaining the operation of the contact coordinate detection unit 1 in step S101 of FIG. FIG. 6A shows an example of a state where the operator's hand is placed on the operation surface of the contact coordinate detection unit 1. FIG. 6B is a diagram in which coordinates (XY coordinates) detected by the contact coordinate detection unit 1 that are orthogonal to each other and reacted independently are represented by diagonal lines. The contact coordinate detection unit 1 outputs the coordinate that has reacted as shown in FIG. 6B to the contact coordinate calculation unit 201 as an X coordinate and a Y coordinate.

  In the present embodiment, the contact coordinate detection unit 1 that detects the fingertip position point of the operator as shown in FIG. 6 is assumed. This is widely used in touch panels and touch screens such as optical type and electrostatic capacity type. The principle of this optical method will be described with reference to the schematic diagram of FIG.

  As shown in FIG. 6C, the optical touch panel has a number of X light emitting units that emit light in a direction that is horizontal to the operation surface and perpendicular to the X coordinate axis, and light emitted by the many X light emitting units. A large number of X light receiving portions for receiving light are provided. Many X light emission parts can be comprised with the row | line | column of many LED, the optical fiber row | line | column etc. which branched the light emitted from one LED. In addition, a large number of X light receiving portions are installed at positions facing the large number of X light emitting portions, and can be constituted by a large number of rows of photodiodes. . Similarly, the optical touch panel has a number of Y light emitting units that emit light in a direction that is horizontal to the operation surface and perpendicular to the Y coordinate axis (perpendicular to the X coordinate axis), and the light emitted by the many Y light emitting units. And a number of Y light-receiving portions that receive light.

  When the finger of the operator touches (or approaches) on the operation surface to shield light emitted from the X light emitting unit and the Y light emitting unit, the X light receiving unit and the Y light receiving unit detect a shadow. Since this shadow appears in both the X coordinate and the Y coordinate, the coordinate touched by the operator can be specified. Here, what is detected by the contact of the finger is an area having a certain width as shown in FIGS. 6 (a) and 6 (b). Each area is a point group detected by several X light receiving units and Y light receiving units, and the contact coordinate detection unit 1 outputs detected coordinates from the point group. As detection coordinates, for example, the coordinates of the center of gravity of an area represented by a point group can be used.

  However, in the case of such a system, when two or more points are touched, a part that is not touched is also detected as a contact coordinate candidate. The method for specifying the coordinates indicated by the fingertip position point at this time will be described together in the detailed description of step S102 and step S103 using FIG.

  FIG. 7 is a flowchart for explaining step S102 and step S103 in FIG. 4 in detail. 8, FIG. 9, FIG. 10, and FIG. 11 are diagrams for explaining the details of each process in FIG.

  First, in step S123-1, the contact coordinate calculation unit 201 acquires the X coordinate and the Y coordinate detected by the contact coordinate detection unit 1, respectively.

  Next, in step S123-2, the contact coordinate calculation unit 201 reads the previous coordinate value stored in the coordinate history storage unit 206. In step S123-3, the contact coordinate calculation unit 201 compares the previous coordinate value read in step S123-2 with the coordinate value acquired from the current contact coordinate detection unit 1, and newly detects the X coordinate. Determine if disappeared or changed.

  Here, the coordinate history storage unit 206 will be described with reference to FIG. The group name is a group of contact information acquired at the same timing, and the related ID creation unit 202 assigns a similar name or number to each contact information acquired at the same timing, and the group name Turn into. The coordinate value represents the coordinate value of the fingertip position point calculated by the related ID creation unit 202. The ID is a number tagged to indicate that the coordinate value is a similar point when continuously moved on the operation surface, and is different when a new coordinate value appears. An ID is assigned by the related ID creation unit 202 and stored in the coordinate history storage unit 206.

  Next, when the contact coordinate calculation unit 201 newly detects an X coordinate in step S123-3, the process proceeds to step S123-4. In step S123-4, the contact coordinate calculation unit 201 sets the newly appearing X coordinate and Y coordinate as the contact coordinates. This case will be described with reference to FIG.

  FIG. 8A shows a state detected by the previous contact coordinate detection unit 1, and FIG. 8C shows information such as coordinates stored in the coordinate history storage unit 206 at this time. FIG. 8B shows a state detected by the contact coordinate detection unit 1 this time.

  In the case of FIG. 8B, the X coordinate detected by the contact coordinate detection unit 1 is x1, x2, and the Y coordinate is y1, y2. In this case, four kinds of coordinates (x1, y1), (x1, y2), (x2, y1), and (x2, y2) are considered as candidates for the coordinates of the fingertip position point. Here, the coordinate history storage unit 206 is referred to, and coordinates other than the previous coordinate value (x1, y1) are set as the coordinates acquired this time. That is, in this example, the contact coordinate calculation unit 201 sets (x2, y2) as the coordinate value of the fingertip position point.

  In addition, if the sampling rate of the contact coordinate detection unit 1 is sufficient (about 20 Hz or more), the operator will consciously detect each time because there is a time difference unless two or more fingers are consciously touched simultaneously. Thus, the coordinate value of each fingertip position point can be calculated.

  Next, in step S123-5, the related ID creation unit 202 assigns a new ID to the coordinate value calculated this time, and further assigns a group name similar to other coordinate values to the coordinate history storage unit 206. Store. FIG. 8D shows information stored in the coordinate history storage unit 206 in FIG.

  Next, the case where the contact coordinate calculation unit 201 detects the disappearance of the X coordinate in step S123-3 will be described. In this case, the process proceeds to step S123-6.

  In step S123-6, the contact coordinate calculation unit 201 extracts a Y coordinate corresponding to the lost X coordinate. This case will be described with reference to FIG.

  FIG. 9A shows the state detected by the previous contact coordinate detection unit 1, FIG. 9C shows information such as the coordinates stored in the coordinate history storage unit 206 at this time, and the current contact coordinate detection unit 1 FIG. 9B shows the state detected in FIG.

  When changing from FIG. 9A to FIG. 9B, the x coordinate x4 disappears in FIG. 9B. At this time, the contact coordinate calculation unit 201 extracts an ID related to x4 stored in the coordinate history storage unit 206. In the case of this example, ID: 4 is extracted with reference to the coordinate history storage unit 206 in FIG.

Next, in step S123-7, the related ID creation unit 202 removes (deletes) information corresponding to the extracted ID, updates other coordinate values, group names, and IDs, and stores them in the coordinate history storage unit 206. Store. FIG. 9D shows each piece of information stored in the coordinate history storage unit 206 in FIG.
As for the relation method between each ID and the coordinate value, the distance between the previous and current coordinate values is calculated, and the previous ID is inherited for the coordinate value having the shortest distance from the previous coordinate value. However, other methods may be used.

  Next, a case will be described in which the contact coordinate calculation unit 201 did not detect the appearance and disappearance of the X coordinate (there was no change) in step S123-3. In this case, the process proceeds to step S123-8.

  In step S123-8, the contact coordinate calculation unit 201 determines whether a new Y coordinate has appeared. At this time, if a new Y coordinate appears, the process proceeds to step S123-9, and if not, the process proceeds to step S123-14. First, a case where a new Y coordinate does not appear will be described with reference to FIG.

  For example, FIG. 10A shows a state detected by the previous contact coordinate detection unit 1, FIG. 10C shows information such as coordinates stored in the coordinate history storage unit 206 at this time, and current contact coordinate detection. When the state detected by the unit 1 is shown in FIG. 10B, the number of X coordinates is not changed, but the number of Y coordinates is reduced.

  In such a case, in step S123-14, the contact coordinate calculation unit 201 calculates the distance between the previous coordinate value and all coordinate candidates detected by the current contact coordinate detection unit 1. In the case of FIG. 10B, all the coordinate candidates are all detected X coordinates (x1 ′, x2 ′, x3 ′, x4 ′) and all Y coordinates (y2 ′, y3 ′, y4 ′). It represents the intersection (12 ways). The distance between all the intersections and the previous coordinate values ((x1, y1), (x2, y2), (x3, y3), (x4, y4)) shown in FIG. 10C is calculated. An example of the calculation result is shown in FIG.

  Next, in step S123-15, the contact coordinate calculation unit 201 counts the number of coordinates of the X coordinate acquired this time. In the case of this example, the acquired X coordinates are x1 ', x2', x3 ', and x4', so there are four. Here, the number of coordinates detected on the X coordinate axis is counted as the number of fingertip positions, and the direction in which the hand is placed in advance is known as shown in FIG. This is because the direction corresponding to the longitudinal direction of the hand is the direction of the Y coordinate axis, and the Y coordinate is easy to overlap because the fingertip positions are close, but the X coordinate value does not overlap from the arrangement of the fingers of the hand. .

  Next, in step S123-16, the contact coordinate calculation unit 201 extracts the coordinates having the shortest distance between the previous and current coordinates by the count number. In the case of the example of FIG. 10, four coordinates are extracted, and the coordinate having the shortest distance from the previous coordinate is (x1) for (x1, y1) from the calculation result of FIG. ', Y3'), (x2, y2) for (x2 ', y2'), (x3, y3) for (x3 ', y3'), (x4, y4) for (x4, y4) x4 ′, y4 ′) are extracted.

  Next, in step S123-17, the related ID creation unit 202 inherits the previous coordinate ID to the paired coordinate values, assigns a group name, and stores the group name in the coordinate history storage unit 206. That is, in the case of the example in FIG. 10, each piece of information stored in the coordinate history storage unit 206 has the result shown in FIG.

  Next, a case where the contact coordinate calculation unit 201 determines in step S123-8 that a new Y coordinate has appeared will be described with reference to FIG.

  For example, FIG. 11A shows the state detected by the previous contact coordinate detection unit 1, FIG. 11C shows information such as the coordinates stored in the coordinate history storage unit 206 at this time, and the current contact coordinate detection. When the state detected by the unit 1 is shown in FIG. 11B, the number of X coordinates does not change, but the number of Y coordinates increases and a new Y coordinate appears.

  In such a case, in step S123-9, the contact coordinate calculation unit 201 calculates the distance between the Y coordinate that appears this time and the Y coordinate of the coordinate value acquired last time. That is, in the case of the example of FIG. 11, the newly appearing Y coordinate is y1 ″ from the difference between FIG. 11A and FIG. 11B, and the Y coordinate of the previously acquired coordinate is FIG. ) To y2 ′, y3 ′, and y4 ′. An example of the calculation result of the distance of the Y coordinate is shown in FIG. In addition, as a method for specifying a newly appearing Y coordinate, the distance between each Y coordinate acquired last time and each Y coordinate acquired this time is calculated, and the Y coordinate whose distance is the shortest based on the previously acquired Y coordinate Are determined in ascending order. In the case of the example in FIG. 11, the number of previous Y coordinates is 3, and the number of Y coordinates acquired this time is 4, so that one Y coordinate is finally added. This Y coordinate is a newly appearing Y coordinate.

  Next, in step S123-10, the contact coordinate calculation unit 201 extracts the X coordinate corresponding to the Y coordinate with the shortest distance from the previously acquired coordinate values with reference to the coordinate history storage unit 206. . In the case of the example of FIG. 11, since the previous Y coordinate having the shortest distance from y1 ″ is y3 ′ from the calculation result of the distance between the Y coordinates in FIG. 2 is extracted as X coordinates corresponding to y3 ′, that is, x1 ′ of ID: 1 and x3 ′ of ID: 3.

  Next, in step S1233-1 and step S123-12, the contact coordinate calculation unit 201 calculates the previous movement amount of the extracted X coordinate, and the X coordinate having the largest movement amount corresponds to the newly appearing Y coordinate. To be extracted as contact coordinates. This presupposes that the direction in which the operator's hand is placed with respect to the contact coordinate detection unit 1 is known in advance by installing the contact coordinate detection unit 1 at the installation position as shown in FIG. That is, when the operator moves his / her finger in the Y-axis direction, attention is paid to the fact that the finger moves not only in the Y-axis direction but also in the X-axis direction in order to move the finger around the wrist. In the case of the example in FIG. 11, when the calculation result of the movement amount of the X coordinate is as shown in FIG. 11F, the X coordinate corresponding to y1 ″ extracted in step S123-9 is x1 ″. It becomes. Accordingly, the contact coordinate calculation unit 201 extracts (x1 ″, y1 ″) as the contact coordinates.

  Next, in step S123-13, the related ID creation unit 202 inherits the ID corresponding to the X coordinate extracted in step S123-12 to the extracted coordinate value, and assigns a group name to the coordinate history storage unit 206. Store. In the case of the example of FIG. 11, an ID corresponding to x1 ′ is assigned to (x1 ″, y1 ″) with reference to the coordinate history storage unit 206 of FIG. Stored in the history storage unit 206. For other coordinates, the same processing as in steps S123-14 to S123-17 is performed, and an ID and a group name are assigned and stored in the coordinate history storage unit 206 (not shown in FIG. 7).

  By performing such processing, the coordinate value that accurately follows the fingertip position point of the operator is identified first even if the contact coordinate detection unit 1 conventionally detects the XY coordinates independently. In addition, even if the operator operates with the finger bent by the method described later, the hand shape of the operator can be faithfully reflected in the hand model.

  FIG. 12 is a flowchart for explaining in detail the processing in step S106 in FIG. First, in step S <b> 106-1, the movement amount calculation unit 203 notifies the operator of a message that prompts the user to open the hand on the operation surface of the contact coordinate detection unit 1 and acquires four initial position points.

  Next, in step S106-2, the movement amount calculation unit 203 uses the initial position point (fingertip position point) acquired in step S106-1 and a model that is the model that the model holding unit 207 holds from the beginning. The initial model is created by deforming the model hand model so that the corresponding fingertip position point of the hand model overlaps. Here, the four initial position points correspond to the little finger, ring finger, middle finger, and index finger, respectively. If the operator's operation is limited to the left hand, use the left hand model hand model so that the initial position point overlaps with the little finger, ring finger, middle finger, and index finger in that order from the left. Deform the model to create an initial hand model.

  Here, when creating the initial hand model, it is necessary to know in which direction the operator is placing his / her hand with respect to the contact coordinate detection unit 1. When the hand is placed in the opposite direction, the initial hand model is created in the opposite direction even at the same four initial position points. Therefore, it is assumed in advance from which direction the operator places his hand with respect to the contact coordinate detection unit (see FIG. 2).

  Thereafter, the movement amount calculation unit 203 displays the created initial hand model on the display unit 3. By this processing, the movement amount calculation unit 203 can create an initial hand model reflecting the size of the operator's hand and display it on the display unit 3.

  Next, in step S106-3, the movement amount calculation unit 203 stores the position of the initial position point acquired in step S106-1 and the initial hand model created in step S106-2 in the model holding unit 207, The process returns to step S102 (see FIG. 4).

  Next, step S108 in FIG. 4 will be described in detail. In step S108, the finger bending angle determination unit 204 calculates the distance between the base point of the hand model and the coordinate value of each fingertip position point, and determines the bending angle of each joint of each finger. Here, the base point of the hand model is a point on the lower palm side and the center of the hand model, which is a reference position of the hand model. FIG. 13 is a diagram illustrating distances R1 to R4 from the base point 1302 of the hand model 1301 to the fingertip position points 1311 to 1314. Since the operator bends or stretches the finger without moving the entire hand, the base point 1302 of the hand model is not moved. The amount of change in length of R1 to R4 shown in FIG. 13 is the amount of movement caused by finger bending.

  Using the amount of movement of the fingertip position point due to finger bending of each finger, the finger bending angle calculation unit 204 determines the bending angle of each joint of each finger. Here, there is an inverse kinematics (hereinafter referred to as IK) technique known in the robot engineering field or the like as a method for obtaining the bending angle of the finger joint from the displacement of the fingertip. The IK technique is used to move the tip of an arm having a plurality of movable parts to a target position. When the arm has a plurality of movable parts, there are a plurality of solutions for the bending angle of the movable part in order to move the tip of the arm to the target position using the IK technique.

  Also in the present invention, since the finger has a plurality of joints, there are a plurality of solutions for the bending angles of the finger joints. For this reason, in the present invention, as an example, using the constraint condition that the palm and fingertip of the operator exist on the operation surface (on the same plane) and the constraint condition that the bending angles of the finger joints are equal, Is uniquely determined.

  FIG. 14A shows the displacement of the fingertip position point when the index finger is bent as an example. FIG. 14B is a side view of the hand model of the index finger 1400 in the bent state shown in FIG. As can be seen from FIG. 14, normally, when the finger is bent, the three finger joints bend at the same time. Therefore, when the hand model is deformed using the constraint condition that the bending angles 1401 to 1403 of the finger joints are equal, the operator feels uncomfortable. The operation can be performed without. Further, when the hand model is deformed using such a constraint condition, the amount of calculation can be greatly reduced, and therefore the hand model that responds instantaneously to the movement of the operator's hand can be deformed.

  Note that the constraint condition for calculating the bending angle of the finger joint is not limited to this, and may be a constraint condition for uniquely obtaining a solution. However, as described above, the constraint condition for calculating the bending angle of the finger joint is preferably a constraint condition that is a hand model that does not cause the operator to feel uncomfortable, and is a hand model that allows the operator to act naturally. Restraint conditions are preferred. In addition, as a method for obtaining the bending angle of the arm movable portion from the target position in the IK technique, a method of repeatedly calculating using a Jacobian matrix, a method of calculating geometrically, and the like are known. May be used.

  The processing in step S108 of FIG. 4 described above is performed, and the finger bending angle calculation unit 204 calculates the bending angle of each finger joint. The finger bending angle calculation unit 204 stores the calculated bending angle of each finger joint at each fingertip position point of the hand model in the model holding unit 207.

  FIG. 15 is a diagram showing a specific example of the deformation of the hand model in step S109 of FIG. FIG. 15A shows an initial position point (a fingertip position point in a state where the operator opens his hand). FIG. 15B is an initial hand model drawn using the initial position points shown in FIG. FIG. 15C shows a fingertip position point when the operator bends the index finger as an example. FIG. 15D is a hand model deformed using the fingertip position points shown in FIG. FIGS. 16A and 16B are views of the hand model shown in FIGS. 15B and 15D as viewed from the side. As shown in FIGS. 15 and 16, in step S <b> 109 of FIG. 4, the hand model is deformed realistically according to the deformation of the operator's hand.

  FIG. 17 is a diagram for explaining the process of step S110 of FIG. As shown in FIG. 17, in step S110, the superimposed image creating unit 209 creates a superimposed image 1720 by superimposing the operation image 1700 on which the GUI parts 1701 are arranged and the deformed hand model 1710, and the display unit 3 To display. Note that when creating the superimposed image 1720, the hand model 1710 may be semi-transparent or the like so that the GUI part 1701 overlapping the hand model 1710 can be easily seen.

  FIG. 18 is a diagram for explaining the process of step S111 of FIG. When the GUI parts 1801 to 1804 and the fingertip position point of the hand model 1810 overlap in the superimposed image, the collision determination unit 210 determines that the GUI part and the fingertip position point of the hand model 1810 have collided. In the case of FIG. 18, the collision determination unit 210 notifies the command transmission unit 211 of the command assigned to the GUI part 1802 for which the collision indicated by the arrow 1820 is determined.

  Thereafter, the command is transmitted to the operation target device 4 in step S112. The operation target device 4 executes the received command.

  As described above, according to the input device according to the present invention, even if each point moves and Y coordinates and the like overlap each other on the touch panel, the subsequent contact coordinates can be correctly extracted. Therefore, when a hand model is created based on the extracted contact coordinates and displayed on the display unit, the user's hand shape can be accurately reflected in the hand model. As a result, the user can perform an operation without feeling uncomfortable.

  In step S104 of FIG. 4 of the present embodiment, four fingertip position points are acquired, but the number is not limited to four. If the right hand or the left hand is known and the finger touching can be identified (for example, touching with the index finger), if the direction of placing the hand with respect to the contact coordinate detection unit 1 is known, the hand model Since the position can be determined, one to three fingertip position points may be acquired.

  In the present embodiment, the fingertip position point is uniquely specified, and then a hand model is created and displayed on the display unit. However, the present invention can also be applied to an input device that does not create / display a hand model. For example, in an input device in which an instruction for operating the operation target device is assigned to each finger in advance and the operator remembers and operates the assigned instruction, it is necessary to display the hand model on the display unit. It is only necessary to uniquely identify the number of fingers operated by the input device.

  According to the input device and the input method of the present invention, even if each point moves and the Y coordinate and the like overlap each other on the touch panel, the subsequent contact coordinates can be correctly extracted. It is useful as an input device for operating various devices.

The figure for demonstrating the outline | summary of operation | movement of the input device 100 which concerns on embodiment of this invention. It is a figure which shows the example of the installation position to the vehicle of the contact coordinate detection part 1, (a) A figure which shows an example when it is attached to the center console of a vehicle, (b) An example when it is attached to the handle of a vehicle The figure to show, (c) The figure which shows an example at the time of being attached to the operation part of the door of a vehicle Functional block diagram of input device 100 concerning an embodiment of the present invention The flowchart for demonstrating operation | movement of the input device 100 which concerns on embodiment of this invention. The figure which shows an example of the hand model which the model holding part 207 hold | maintains FIG. 5 is a diagram for explaining the operation of the contact coordinate detection unit 1 in step S <b> 101 of FIG. 4, (a) a diagram illustrating an example of a state where an operator's hand is placed on the operation surface of the contact coordinate detection unit 1; FIG. 4 is a diagram showing fingertip position points detected by the contact coordinate detection unit 1, and FIG. 4C is a schematic diagram for explaining the operation principle of the optical contact coordinate detection unit 1. Flowchart for explaining in detail the processing of steps S102 and S103 in FIG. FIGS. 7A and 7B are diagrams for explaining the processing of steps S123-4 and S123-5 in FIG. 7, (a) a diagram showing a state detected by the previous contact coordinate detection unit 1, and (b) detection by the current contact coordinate detection unit 1; The figure which shows the state performed, (c) The figure which shows each information stored in the coordinate history storage part 206 in the last detection state, (d) Each information stored in the coordinate history storage part 206 in this detection state Figure showing FIGS. 7A and 7B are diagrams for explaining the processing of steps S123-6 and S123-7 in FIG. 7, (a) a diagram showing a state detected by the previous contact coordinate detection unit 1, and (b) detection by the current contact coordinate detection unit 1; The figure which shows the state performed, (c) The figure which shows each information stored in the coordinate history storage part 206 in the last detection state, (d) Each information stored in the coordinate history storage part 206 in this detection state Figure showing FIGS. 7A and 7B are diagrams for explaining the processing of steps S123-14 to S123-17 in FIG. 7, (a) a diagram showing a state detected by the previous contact coordinate detection unit 1, and (b) detection by the current contact coordinate detection unit 1; The figure which shows the state performed, (c) The figure which shows each information stored in the coordinate history storage part 206 in the last detection state, (d) Each information stored in the coordinate history storage part 206 in this detection state The figure which shows (e) The figure which shows an example of the calculation result of each distance of the previous coordinate value and all the coordinate candidates detected this time FIGS. 7A and 7B are diagrams for explaining the processing of steps S123-9 to S123-13 in FIG. 7, (a) a diagram showing a state detected by the previous contact coordinate detection unit 1, and (b) detection by the current contact coordinate detection unit 1; The figure which shows the state performed, (c) The figure which shows each information stored in the coordinate history storage part 206 in the last detection state, (d) Each information stored in the coordinate history storage part 206 in this detection state The figure which shows (e) The figure which shows an example of the calculation result of the distance between the Y coordinate which newly appeared this time, and the Y coordinate acquired last time, (f) The movement amount between the last time and this time X coordinate in the extracted X coordinate Of an example of the calculation result Flowchart for explaining in detail the process of step S106 of FIG. The figure which shows distance R1-R4 from the base point 1302 of the hand model 1301 to each fingertip position point 1311-1314 FIG. 5 is a diagram for explaining a method of calculating a finger bending angle of a hand model in step S108 of FIG. 4, (a) a diagram illustrating displacement of a fingertip position point when the index finger 1400 is bent, and (b) a bent state. View of hand model of index finger 1400 from the side FIG. 5 is a diagram illustrating a specific example of the hand model deformation in step S109 of FIG. 4, (a) a diagram illustrating an initial position point, (b) a diagram illustrating an initial hand model drawn using the initial position point, and (c). The figure which shows the fingertip position point when an operator bends the index finger, (d) The figure which shows the hand model deform | transformed using the fingertip position point when the index finger is bent 15 is a view of the hand model shown in FIG. 15 viewed from the side, (a) a view of the hand model shown in FIG. 15b as viewed from the side, and (b) a view of the hand model shown in FIG. 15d as viewed from the side. The figure for demonstrating the process of step S110 of FIG. The figure for demonstrating the process of step S111 of FIG. The figure explaining the candidate of the coordinate detected in an optical touch panel The figure for demonstrating an example when a contact coordinate cannot be specified in an optical touch panel

DESCRIPTION OF SYMBOLS 1 Contact coordinate detection part 2 Operation part 3 Display part 4 Operation object apparatus 100 Input device 201 Contact coordinate calculation part 202 Related ID preparation part 203 Movement amount calculation part 204 Finger bending angle calculation part 205 Model shape determination part 206 Coordinate history storage part 207 Model holding unit 208 GUI part holding unit 209 Superimposed image creating unit 210 Collision determining unit 211 Command transmitting unit 1701, 1801-1804 GUI parts 1710, 1810 Hand model 1720 superimposed image

Claims (4)

An input device that performs an input operation by specifying a contact position of a user's hand from a detection position on an operation surface of a touch panel,
When the coordinates on the operation surface are expressed by the X coordinate axis and the Y coordinate axis orthogonal to each other, and the direction corresponding to the longitudinal direction of the user's hand is the Y coordinate axis, the number of coordinates detected on the X coordinate axis is detected on the Y coordinate axis. An input device characterized in that the number of coordinates detected on the X coordinate axis is set as the number of contact points of the user's hand when the number of coordinates is small. When the number of coordinates detected on the Y coordinate axis is one less than the number of coordinates detected on the X coordinate axis, if a new coordinate is detected on the Y coordinate axis, the distance of the Y coordinate of the detection position before and after the detection Based on the above, the two detection positions where the Y coordinate is overlapped are specified, and the detection position with the larger change in the X coordinate is specified as the contact position of the user's hand among the two detection positions before and after the detection. The input device according to claim 1. An input method for performing an input operation by specifying a contact position of a user's hand from a detection position on an operation surface of a touch panel,
When the coordinates on the operation surface are expressed by the X coordinate axis and the Y coordinate axis orthogonal to each other, and the direction corresponding to the longitudinal direction of the user's hand is the Y coordinate axis, the number of coordinates detected on the X coordinate axis is detected on the Y coordinate axis. An input method comprising the step of setting the number of coordinates detected on the X coordinate axis as the number of contact points of the user's hand when the number of coordinates to be detected is small. When the number of coordinates detected on the Y coordinate axis is one less than the number of coordinates detected on the X coordinate axis, if a new coordinate is detected on the Y coordinate axis, the distance of the Y coordinate of the detection position before and after the detection Based on the above, the step of identifying two detection positions where the Y coordinate overlapped, and the detection position with the larger change in the X coordinate of the two detection positions before and after the detection as the contact position of the user's hand 4. The input method according to claim 3, further comprising a specifying step.

Images