JP2010117748A - Input device and input method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an input device and an input method for determining whether a right or left hand of a user is used using only a contact information detection part which can be operated by a plurality of fingers of the user. <P>SOLUTION: The input device configured to operate equipment with fingers placed on an operation face includes: a contact information detection part for detecting a plurality of coordinates of contact between the operation face and the plurality of fingers; a contact information history storage part for storing the detected contact coordinates; a related ID creation part for assigning ID to each stored contact coordinate; a gesture recognition part for recognizing a gesture on the basis of the history of each contact coordinate with the same ID assigned thereto; a right/left hand determination part for determining whether the finger placed on the operation face belongs to the right hand or the left hand on the basis of each contact coordinate stored in the contact information history storage part; and a command transmission part for transmitting a predetermined command to the equipment on the basis of the recognition result of the gesture recognition part and the determination result of the right/left hand determination part. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an input device and an input method for a user to input a command or the like to a device, and more specifically, using a body part such as a hand based on information displayed on a display or the like by a user. The present invention relates to an input device and an input method capable of inputting commands and the like.

As a device for operating devices such as a car navigation system, a mobile phone, a personal computer, and a digital camera, a touch panel that detects a position touched by a user is generally widespread. In recent years, devices capable of detecting a plurality of touching points have been gradually installed in various devices (for example, Patent Document 1).
Among them, there is a device in which a touch panel is mounted on a digital camera, and the menu position on the touch panel is changed by discriminating the left and right hands of the user (see, for example, Patent Document 2).

This is because the user can operate the touch panel by operating the touch panel by determining which hand is operated by the user using either the left hand or the right hand and using the sensor disposed on the shutter. The menu is displayed near.
Japanese Patent Publication No. 7-82419 JP 2004-340991 A

  However, the prior art of Patent Document 2 has a problem in that it is necessary to provide a separate sensor for discriminating the left and right hands, which increases costs.

  The present invention has been made in order to solve the above-described conventional problems. An input device and an input method that can specify the left and right hands of a user only with contact information on the touch panel when operating the touch panel. The purpose is to provide.

  A first aspect of the present invention is an input device that operates a device with a finger placed on an operation surface, a contact information detection unit that detects a plurality of contact coordinates between the operation surface and the plurality of fingers, and contact information The contact information history storage unit that stores the contact coordinates detected by the detection unit, the related ID creation unit that assigns an ID to each contact coordinate stored in the contact information history storage unit, and the same ID by the related ID creation unit And a gesture recognition unit that recognizes a gesture based on the history of each contact coordinate, and a finger placed on the operation surface based on each contact coordinate stored in the contact information history storage unit A left and right hand determination unit that determines whether the object is a thing, and a command transmission unit that transmits a predetermined command to the device based on the recognition result by the gesture recognition unit and the determination result of the left and right hand determination unit .

  The left and right hand determining unit determines the left and right hands based on the relative positional relationship of the plurality of contact coordinates.

  Further, the contact information detection unit detects a contact area in addition to a plurality of contact coordinates between the operation surface and a plurality of fingers, and a contact information history storage unit stores the contact coordinates and the contact area detected by the contact information detection unit, The left and right hand determination unit determines the left and right hands based on the relative positional relationship and contact area of each contact coordinate stored in the contact information history storage unit.

  The gesture recognizing unit further calculates the movement direction of each contact coordinate based on the history of each contact coordinate given the same ID by the related ID creating unit, and the left and right hand determining unit is calculated by the gesture recognizing unit. The left and right hands are determined based on the movement direction of the plurality of contact coordinates.

  The gesture recognizing unit further calculates the movement direction of each contact coordinate based on the history of each contact coordinate given the same ID by the related ID creating unit, and the left and right hand determining unit is calculated by the gesture recognizing unit. The left and right hands are determined based on the movement direction of the plurality of contact coordinates and the relative positional relationship of the plurality of contact coordinates.

  The left and right hand determination unit further calculates the movement amounts of the plurality of contact coordinates, and determines the left and right hands based on the calculated movement directions and movement amounts of the plurality of contact coordinates.

  Further, the contact information detection unit detects a contact area in addition to a plurality of contact coordinates between the operation surface and a plurality of fingers, and a contact information history storage unit stores the contact coordinates and the contact area detected by the contact information detection unit, The left and right hand determination unit determines the left and right hands based on the movement direction of the plurality of contact coordinates calculated by the gesture recognition unit and the contact area stored in the contact information history storage unit.

  The second aspect of the present invention is directed to an input method for operating a device with a finger placed on an operation surface. And in order to achieve the said objective, the input method of this invention stores the contact information detected by the contact information detection step which detects several contact coordinates of an operation surface and several fingers, and the contact information detection step. The contact information history storing step, the related ID creating step for assigning an ID to each contact coordinate stored in the contact information history storing step, and the history of each contact coordinate assigned with the same ID by the related ID creating step A gesture recognition step for recognizing a gesture based on the left hand and a left hand determination step for determining whether a finger placed on the operation surface is a right hand or a left hand based on each contact coordinate stored in the contact information history storage step And a predetermined command based on the recognition result in the gesture recognition step and the determination result in the left and right hand determination step. And a command transmission step of transmitting to the vessel.

  According to the input device and the input method of the present invention, it is possible to determine the left and right hands of the user without providing a separate sensor. Therefore, the application can be operated according to the left and right hands without increasing the cost. Have

  Hereinafter, an input device and an input method according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram illustrating a hardware configuration example of an input device 100 according to an embodiment of the present invention. In FIG. 1, the input device 100 includes a contact information detection unit 1 and a calculation unit 2. The calculation unit 2 includes an I / O 11 that inputs and outputs data, an HDD (Hard Disk Drive) 12 that stores data, programs, and the like, a ROM 13 that stores basic programs such as BIOS, and programs and programs stored in the HDD 12. A CPU 14 that executes a program stored in the ROM 13, a RAM 15 that holds data, an image processor 17 that processes image data, a VRAM 18 that holds image data, and a bus line 16 that connects these components are provided. .

  The contact information detection unit 1 is connected to the input side of the I / O 11 and is a touch panel or touch screen that can detect the coordinates and area touched by a plurality of fingers of the user (hereinafter referred to as contact information by combining the coordinates and area). Yes, as a method, a capacitance type, a resistance film type, an optical type, or the like can be used. The contact information input from the contact information detection unit 1 to the input side of the I / O 11 is held in the RAM 15.

  In the case where the contact information detection unit 1 is an optical touch panel, for example, as shown in FIG. 2, the contact position is independently determined on the X axis and the Y axis, and thus there may be cases where the contact coordinates cannot be detected uniquely. To do. This is because in FIG. 2, although the point A (x1, y2) and the point B (x2, y1) are actually touching, another point C (x1, y1) and point D (x2, y2) This is because is detected. In such a case, if the sampling rate at the time of coordinate detection is sufficient (about 20 Hz or more), the user can consciously detect each of them because a time difference occurs unless two or more points are consciously touched simultaneously. . For example, suppose that point A first touches and then point B touches. Here, first, the detected coordinates (x1, y2) are stored as the coordinates of the point A. After that, at the next timing, the points B (x2, y1), C (x1, y1), and D (x2, y2), which are three new coordinates, are detected. It can be estimated that the points except (x1, y2) should be newly detected by the touch of the user's finger. Thereby, (x2, y1) is stored as the coordinates of the point B where the user touches. By performing such processing, two or more coordinates can be easily detected.

  Returning to FIG. 1, the display unit 3 is connected to the output side of the I / O 11. The display unit 3 is a display that displays an image corresponding to the image data. Further, an operation target device 4 such as a car navigation device, an audio device, and an air conditioner operated by the input device 100 is connected to the output side of the I / O 11. The calculation unit 2 outputs an instruction to the operation target device 4 based on the user instruction detected by the contact information detection unit 1.

  The HDD 12 determines and transmits a command to be sent to the operation target device 4 based on a user input to the contact information detection unit 1, or causes the display unit 3 to display a GUI for operating the operation target device 4. A control program for causing the CPU 14 to function is stored.

  Note that the control in the input device 100 is executed by hardware or the cooperation of hardware and software.

  Hereinafter, the configuration and operation of the input device 100 will be described in more detail. First, the configuration of the input device 100 will be described. FIG. 3 is a diagram illustrating functional blocks of the input device 100. As illustrated in FIG. 3, the input device 100 includes a contact information detection unit 1 and a calculation unit 130.

  The calculation unit 130 includes a related ID creation unit 131, a contact information history storage unit 132, a gesture recognition unit 133, a left / right hand determination unit 134, and a command transmission unit 135.

  The related ID creation unit 131 calculates the relevance of the contact information currently acquired from the past contact information, assigns an ID to each contact information, and stores the current contact information in the contact information history storage unit 132 again. To do. This relationship indicates which contact information of the past contact information corresponds to the contact information detected at present.

  The contact information history storage unit 132 stores the coordinates up to the present and the history of the contact area in association with each other.

  The gesture recognition unit 133 recognizes the user's gesture from the contact information history stored in the contact information history storage unit 132.

  The left and right hand determining unit 134 determines which of the left and right hands the user has operated from the recognized gesture.

  The command transmission unit 135 determines a command to be transmitted to the control target device 4 from the discrimination result of the left and right hands.

  Next, the overall processing flow of the input device configured as described above will be described with reference to the flowchart of FIG.

  First, in step S101, the contact information detection unit 1 detects the coordinates and area (contact information) of a user's finger that touches the contact information detection unit 1, and outputs them. A method for acquiring the coordinates and area of the contact information detection unit 1 will be described later.

  Next, in step S <b> 102, the related ID creation unit 131 associates the contact information acquired by the contact information detection unit 1 with the past contact information and the currently acquired contact information. First, the related ID creation unit 131 refers to the contact information history storage unit 132 and calculates the relevance of the contact information acquired from the past contact information. A method for calculating this relationship will be described later. Next, the related ID creating unit 131 assigns an ID representing the relationship with the past contact information to the contact information detected this time, and stores it in the contact information history storage unit 132.

  Next, in step S103, the gesture recognition unit 133 refers to a plurality of contact information histories from the contact information history storage unit 132, and from the contact information history, the movement direction, movement amount, and appearance of each contact information associated with each ID. / Recognize various gestures using disappearance patterns. A method for recognizing this gesture will be described later.

  Next, in step S104, the left / right hand determining unit 134 determines whether the operation by the user's finger is performed with the right hand or the left hand based on the moving direction, moving amount, appearance / disappearance pattern, etc. of the contact information associated with each ID. The determination result is output to the command transmission unit 135. This left and right hand determination method will be described later.

  In step S <b> 105, the command transmission unit 135 transmits various parameters to the control target device 4 based on the input gesture result and the left and right hand determination results.

  For example, as shown in FIG. 5, the gesture recognition unit 133 recognizes the gesture of opening and closing two points of the thumb and the index finger with the contact information detection unit 1, and the right and left hand determination unit 134 is an operation with the right hand. Assume that a command for changing the sound volume is assigned. In this case, a volume parameter is transmitted to the control target device 4 according to the distance between the coordinates of the two points in contact.

  As another example, in car navigation or the like that partially restricts / limits the input to the control target device 4 of the driver who is traveling in order to ensure the safety of vehicle driving, the contact information detection unit 1 is shown in FIG. ) Is attached to the position of the center console as shown in FIG. In this case, the operation from the driver's seat is necessarily operated with the left hand, and the operation from the passenger seat is performed with the right hand. Utilizing this property, for example, a gesture as shown in FIG. 5 is assigned to a command for changing the scale of a map for car navigation, and this is recognized, and the left / right hand determining unit 134 determines that the operation is a left hand operation. In this case, it is estimated that the user who is operating is the driver. Then, the command transmission unit 135 changes the scale parameter for the first time when the scale parameter changes from the state of FIG. 5A to the state of FIG. 5B.

  Similarly, when the right hand is detected, it is presumed that the operation is from the passenger seat, and the parameter is finely changed according to the distance between the two touching points. As a result, when the driver operates during driving, no detailed operation is performed, so that the driver can concentrate on the driving operation and can safely operate the device to be controlled. Further, in the operation of the passenger seat, a fine operation can be performed with one gesture and an intuitive operation can be performed. When the right seat is the passenger seat and the left seat is the driver seat, the parameters output to the control target device 4 from the left and right hand judgment results are opposite.

  If the left and right hand determination unit 134 cannot determine the left and right hands, a display using a pop-up or a voice for transmitting an error is used to notify the user that the determination of the left and right hands is impossible. It may be output.

  In the present embodiment, it is necessary to know from which direction the operator has entered the hand with respect to the contact information detection unit 1. Therefore, it is assumed in advance from which direction the operator enters the hand with respect to the contact information detection unit 1.

  FIG. 6 shows an example of the position where the contact information detector 1 is installed in the vehicle. FIG. 6A is a diagram showing an example when the vehicle is attached to the center console of the vehicle. FIG. 6B is a diagram illustrating an example when the contact information detection unit 1 is attached to the steering wheel of the vehicle. FIG.6 (c) is a figure which shows an example at the time of the contact information detection part 1 being attached to displays, such as a car navigation. By attaching the contact information detection unit 1 to the position of the vehicle as shown in FIG. 6A, it is known in advance that the user who is a driver makes the left hand enter the direction of the arrow 601 with respect to the contact information detection unit 1. The In addition, by attaching to the position of the vehicle as shown in FIG. 6B, it is known in advance that the user who is the driver makes the left hand enter the direction of the arrow 602 with respect to the contact information detection unit 1. In addition, by attaching to the vehicle position as shown in FIG. 6C, it is known in advance that the user who is a driver causes his / her left hand to enter the direction of the arrow 603 with respect to the contact information detection unit 1.

  Next, each step described with reference to FIG. 4 will be described in detail.

  FIG. 7 is a diagram for explaining the operation of the contact information detection unit 1 in step S101 of FIG. FIG. 7A shows an example of a state where the operator's hand is placed on the operation surface of the contact information detection unit 1. FIG. 7B is a diagram illustrating the contact point group detected by the contact information detection unit 1 as a black circle group. As shown in FIG. 7, the contact information detection unit 1 detects a contact area of a finger by detecting a contact point (contact point group) in step S <b> 101 of FIG. 4.

  FIG. 8 is a flowchart for explaining step S101 of FIG. 4 in detail. FIG. 9 is a diagram for explaining the contact coordinates acquired in step S101 of FIG. Hereinafter, the processing in step S101 in FIG. 4 will be described in detail with reference to FIGS.

  First, in S101-1, the contact information detection part 1 acquires the data which show the contact point group demonstrated using FIG.7 (b).

  Next, in step S101-2, the contact information detection unit 1 calculates a plurality of regions indicated by ellipses in FIG. 9 based on the positional relationship between the contact points. A general method may be used as a method of calculating a plurality of regions in S101-2. For example, when there are other contact points in the vicinity of the contact point, there is a method of dividing these contact points into a plurality of regions by regarding them as the same region.

  Next, in step S101-3, the contact information detection unit 1 performs an ellipse fitting process on each area calculated in step S101-2. FIG. 9 is a diagram for explaining a method of calculating contact coordinates. In step S101-3, the contact information detection unit 1 encloses the contact point group of the finger with an ellipse as indicated by an ellipse A in FIG. Thereafter, the contact information detection unit 1 uses, for example, the center of the ellipse A as the contact coordinates (see FIG. 9B). The contact coordinates may be determined at other positions using other methods.

  After determining the contact coordinates, the contact area is obtained for the contact coordinates. The contact area is an area where each fingertip is in contact with the contact information detection unit 1. For example, when the contact coordinates are obtained by ellipse fitting as described above, the contact area can be obtained by calculating the area of the ellipse. Further, the number of contact points in each region can be used as the contact area.

  Next, in step S101-4, the contact information detection unit 1 stores the contact coordinates and the contact area acquired in step S101-3 in the contact information history storage unit 132. FIG. 10 is a diagram illustrating an example of the position of the contact coordinates acquired in step S101-3 in FIG. In FIG. 10, black circles indicate contact coordinates 1001, and dotted ellipses 1002 indicate contact area regions.

  The flow of processing of the related ID creation unit in step S102 in FIG. 4 will be described with reference to FIGS.

  Here, the process of step S102 this time will be described using an example in which all the fingertips (5 points) of the user's one hand are in contact with the contact information detection unit 1.

  FIG. 12A shows a state in which the previous user has touched the contact information detection unit 1, and FIG. 12B shows each point stored in the contact information history storage unit 132 at this time. Represents information. That is, for each point (point A, point B, point C, point D, point E) in FIG. 12A, the point A is ID “1”, the point B is ID “2”, and the point C is ID “3”. ", Point D is stored in association with ID" 4 ", point E is associated with ID" 5 ", and the same group name" 10 "is assigned. This group name is a group of contact information acquired at the same timing, and the related ID creation unit 131 gives a similar name or number to each contact information acquired at the same timing, Group.

  Next, assuming that contact information of each point is stored in the contact information history storage unit 132 as shown in FIG. 12B last time, a detailed flow of the processing of S102 will be described with reference to FIG.

  First, in step S102-1, the related ID creation unit 131 determines whether contact information is input from the contact information detection unit 1. When the contact information is acquired, the process proceeds to step S102-2, and when no contact information is input (when no finger is in contact with the contact information detection unit 1), the process proceeds to step S102-. 13, the related ID creation unit 131 assigns only the group name and stores it in the contact information history storage unit 132. Thereby, it can represent that it is not contacting.

  Next, in step S102-2 and step S102-3, the related ID creation unit 131 refers to the coordinate value of each previous contact information stored in the contact information history storage unit 132, and each coordinate at the previous timing. Is stored in the contact information history storage unit 132. When contact information of 5 points is stored as shown in FIG. 12B, the process proceeds to step S102-4, and the contact information is stored because the previous user has not touched the contact information detection unit 1. If not, the process proceeds to step S102-11.

  Next, in step S102-4, the related ID creation unit 131 calculates the distance between the coordinates of the previous contact information and the contact information detected this time. For example, as shown in FIG. 13A, each point (point A ′, point B ′, point E) detected this time from each point (point A, point B, point C, point D, point E) detected last time. C ′, point D ′, point E ′). Here, the contact information of each point detected this time is shown in FIG. At this time, for example, the distance between each point acquired this time and the previous time is calculated as shown in FIG.

  Next, in step S102-5, the related ID creating unit 131 extracts the previous contact information with the shortest distance as a set of two distances among the coordinates calculated in S102-4. In the case of the result as shown in FIG. 13C, the point having the shortest distance from the currently acquired point A ′ is the point A, the point B ′ is the point B, the point C ′ is the point C, and the point D. 'Is point D and point E' is extracted as a pair of points E.

  Next, in steps S102-6 and S102-7, the related ID creation unit 131 compares the number of coordinates of the contact information acquired last time and this time, and changes the process of shifting according to the number of coordinates. In the case of FIG. 13, since the contact information detection unit 1 has detected 5 points last time and 5 points this time, the number is the same, and the process proceeds to step S102-8. Other cases will be described later.

  Next, in step S <b> 102-8, the related ID creation unit 131 inherits the previous contact information ID corresponding to the contact information detected this time and stores it in the contact information history storage unit 132. That is, the inherited ID is determined from the set extracted in step S102-5 and stored. In the case of FIG. 13, since the previous contact information corresponding to the point A ′ is the point A, the ID “1” of the point A is inherited to the point A ′. The ID is similarly inherited for other points.

  Next, in step S <b> 102-12, the related ID creation unit 131 assigns the same group name to each contact information acquired this time and stores it in the contact information history storage unit 1. Thereafter, the process proceeds to step S103.

  In step S102-7, when the current number of coordinates acquired by the contact information detection unit 1 is smaller than the previously acquired number of coordinates, the process proceeds to step S102-9.

  Next, in step S102-9, for each contact information acquired this time, the related ID creation unit 131 extracts the previous contact information for which the distance becomes shorter, and the ID of the previous contact information for the contact information acquired this time is extracted. It is inherited as an ID and stored in the contact information history storage unit 1. Here, as shown in FIG. 14A, the previous contact information is 5 points (point A, point B, point C, point D, point E), and the contact information input this time is 4 points (point A). ', Point B', point D ', point E') will be described as an example. It is assumed that the distance between each coordinate acquired last time and this time is a value as shown in FIG. In FIG. 14B, the distance from the point A ′ is the shortest at the point A, the point B ′ is the point B, the point D ′ is the point D, and the point E ′ is the point E. The point A ′ inherits the ID of the point A, and the other points B ′, D ′, and E ′ also inherit the corresponding IDs and store them in the contact information history storage unit 132. Thereafter, the process proceeds to step S102-12.

  In step S102-7, when the current number of coordinates acquired by the contact information detection unit 1 is larger than the number of coordinates acquired last time, the process proceeds to step S102-10.

  Next, in step S102-10, the related ID creation unit 131 sets the previous contact information ID for the current contact in order from the shortest distance between each contact information acquired this time and each contact information acquired last time. The information is inherited and stored in the contact information history storage unit 132. Here, as shown in FIG. 15A, the previous contact information is 4 points (point A, point B, point D, point E), and the contact information input this time is 5 points (point A ′, point A). A case of B ′, point C ′, point D ′, point E ′) will be described as an example. It is assumed that the distance between each coordinate acquired last time and this time is a value as shown in FIG. As shown in FIG. 15B, the shortest distance between two points is “1”, a point A ′ is a point A, a point E ′ is a point E, and a pair of points A and E is extracted. The ID is inherited and saved at points A ′ and E ′, respectively. Next, the short distance is “2”, the point B ′ inherits and stores the ID of the point B, and the point D ′ inherits the ID of the point D.

  Next, in step S102-11, the related ID creation unit 131 assigns a new ID to the contact information to which no ID is assigned among the contact information detected by the contact information detection unit 1 this time, and stores the contact information history. Stored in the unit 132. In the case of FIG. 15, since the contact information whose ID has not been verified in step S102-10 is the point C ', the related ID creation unit 131 assigns a new ID to the point C' and stores it. Thereafter, the process proceeds to step S102-12.

  Next, the gesture recognition process in step S103 and the left and right hand determination process in step S104 will be described in detail. Here, these processes are divided into several methods, and each method will be described using a flowchart or the like.

(1) Case of discriminating left and right hands from static relative position The flowchart of FIG. 16 determines the user's left and right hands from the relative position of each coordinate of the contact information when the user touches the contact information detecting unit 1 with a finger. The flow of processing when doing this is shown.

  First, in step S103-1a, the gesture recognition unit 133 acquires the current mode from the control target device 4 to be controlled. This mode represents what screen is being displayed, what function is being executed, or what function can be executed. For example, when the control target device 4 displays a map screen by car navigation, functions such as map scrolling, scale change, and destination setting can be executed. Here, the acquired mode in FIG. 16 is a mode in which the scale of the map can be changed on the map screen. In order to execute this function, it is assumed that the scale can be changed by bringing the forefinger and the thumb into contact with the contact information detection unit 1 and opening and closing each other as shown in FIG.

  Next, in S103-2a, the gesture recognition unit 133 acquires the past several pieces of contact information from the latest contact information for each group name from the contact information history storage unit 132. The number of contact information acquired at this time depends on the sampling rate at which the contact information detection unit 1 acquires the contact information. For example, if the sampling rate of the contact information detection unit 1 is 20 Hz, each contact information is acquired 20 times per second and stored in the contact information history storage unit 132, respectively. Therefore, when it is desired to acquire contact information for one second, it is sufficient to acquire 20 pieces of contact information.

Next, in S103-3a, the gesture recognizing unit 133 determines whether or not the same number of contact information exists within a predetermined time. For example, assuming that 10 groups (0.5 seconds) have been acquired from the contact information history storage unit 1, it is determined whether two points of contact information exist continuously for each group. At this time, if there is a point other than 2 points (1 point or 3 points) in one group, the process proceeds to step S101. If the same number exists, the process proceeds to step S103-4a.
Next, in S103-4a, the gesture recognition unit 133 determines whether the movement amount of each contact information is within a threshold value. For example, referring to the IDs of the acquired contact information of the 10 groups, it is determined whether or not the movement amount of the contact information with the same ID is within a threshold. That is, it is determined whether or not each coordinate has moved significantly for a certain period of time (in this example, 0.5 seconds). When the gesture recognizing unit 133 determines that each coordinate has moved more than a predetermined threshold, the process proceeds to step S101, and when it is determined that the coordinate is within the predetermined threshold, the process proceeds to step S104-1a.

  Next, in step S104-1a, the left / right hand determining unit 134 refers to the coordinate value of the contact information stored most recently among the acquired contact information, and determines the left / right hand from each relative position. When the left and right hands are determined from the relationship between the relative positions, attention is paid to the fact that the Y coordinate of the two points (the coordinate system in FIG. 17) is the thumb. This is because when the operation as shown in FIG. 5 is performed, the Y coordinate value of the thumb is usually smaller than the Y coordinate value of the index finger.

  Next, a method for determining the left and right hands will be described with reference to FIGS.

  FIG. 17A shows that the X coordinate of the other point B is larger than the X coordinate of the point A having a small Y coordinate as a result of comparing the Y coordinates of the two points A and B. In this case, since another point is detected as contact information on the right side of the thumb (the X coordinate is large), the left / right hand determining unit 134 determines that the hand is a right hand when the operation shown in FIG. 5 is performed. In FIG. 17B, the X coordinate of the other point A is smaller than the X coordinate of the point B having a small Y coordinate, so the left / right hand determining unit 132 determines that the hand is a left hand. It should be noted that, as shown in FIG. 18 (a), when the X coordinates of the two points are coincident or the distance between the X coordinates is equal to or smaller than the threshold value, or when the Y coordinates of the two points are coincident or as shown in FIG. If the distance between the two is less than or equal to the threshold value, the left and right hands cannot be determined.

  As described above, it is not necessary to move the finger specially in order to discriminate between the left and right hands by discriminating between the left and right hands without moving the fingers in the state of being in contact with the contact information detection unit 1, and processing load is not applied. It becomes possible to distinguish left and right hands.

  Further, in the processing flow of FIG. 16, the acquired mode may be a mode only for determining the left and right hands of the user. Furthermore, in the above example, the processing has been described with an example using the coordinates of the contact information of two points, but the contact information may be used up to five points.

  For example, when the control target device 4 prompts the contact information detection unit 1 to touch each fingertip with all fingers extended in order to determine the left and right hands, the left and right hand determination unit 134 The left and right hands can be discriminated as follows.

  This case will be described with reference to FIG. As shown in FIG. 19A, when the left and right hand determining unit 134 determines the left and right hands using the contact information of five points, the point (point A) having the smallest coordinate value of the X coordinate among all the points is selected. If the Y coordinate is the smallest compared to other points (in this case, point B, point C, point D, point E), this is determined to be the right hand. Further, as shown in FIG. 19B, the Y coordinate of the point (point E) having the largest X coordinate value among all the points is changed to other points (in this case, point A, point B, point If it is the smallest compared to C, point D), this is determined as the left hand. This is because the coordinates where the thumb is located are the smallest in the Y coordinate compared to the other fingers when the finger touches the contact information detection unit 1 with the finger extended. Due to this property, the right hand is touched when the point with the smallest Y coordinate is on the left side of the other coordinates (the smallest X coordinate), and the left hand finger is touched when the point is on the right side (the largest X coordinate). It is thought that.

(2) Case of discriminating left and right hands from static relative position + contact area Although not described in the flowchart of FIG. 16, in order to further ensure the judgment result of left and right hands, the contact area of each contact information is defined as a condition. Can be used together. This focuses on the fact that when a human finger comes into contact with something, it is the thumb that has the largest contact area. In such processing, in step S104-1a or subsequent processing, the left and right hand determination unit 134 compares the contact area of each point and determines the left and right hands.

  In the case of FIG. 19A, it is determined whether the contact area of the point A located on the thumb, and in the case of FIG. 19B, the contact area of the point E is larger than the contact area of other points, and the left and right hands are determined. If the contact area of these points is large, the right hand is determined in FIG. 19A and the left hand is determined in FIG. 19B.

  Thereby, when the user wants to reverse the determination of the left and right hands, if the user intentionally turns over and touches the contact information detection unit 1, the relative relationship of the coordinate values of each contact information is reversed on the left and right, but the contact area It is possible to prevent this by using as a determination condition.

(3) When discriminating the left and right hands from the moving direction The flowchart in FIG. 20 is a process in which the user's left and right hands are determined based on the moving direction in which the user moves the finger in the state where the finger is in contact with the contact information detection unit 1. Shows the flow. Also in this case, as in FIG. 16, the mode acquired from the control target device 4 is a mode in which the scale of the map can be changed on the map screen. In order to execute this function, as shown in FIG. However, it is assumed that the scale can be changed by bringing the forefinger and the thumb into contact with the contact information detection unit 1 and opening and closing each other's fingers. The flow of each process will be described using such an example.

  First, in steps S103-1b to S103-3b, the gesture recognition unit 133 performs the same processing as in steps S103-1a to S103-3a.

  Next, in step S103-4b, the gesture recognition unit 133 calculates the movement direction of each coordinate from each contact information history. This moving direction can be calculated by subtracting the latest contact information from the coordinate values of the past contact information in the acquired contact information.

  For example, the contact information of the past two points (point A and point B) is as shown in FIG. 21A, and the latest contact information of the two points (point A ′ and point B ′) is shown in FIG. In the state shown in (b), the movement direction in the X-axis direction from the point A to the point A ′ is (x1′−x1) <0, and the movement direction in the Y-axis direction is (y1′−y1). <0, the movement direction in the X-axis direction from the point B to the point B ′ is (x2′−x2)> 0, and the movement direction in the Y-axis direction is (y2′−y2)> 0. Here, it is assumed that point A and point A ', and point B and point B' have the same ID.

  Next, in step S103-5b, the gesture recognition unit 133 calculates parameters. Here, when the acquired mode is a mode in which the scale of the map is changed, the change rate of the distance between the thumb that has touched the contact information detection unit 1 and the index finger is used as a scale parameter. Note that the initial distance is the distance between the coordinates at which the two points touched first, and if the distance increases from there, it has a positive parameter, and further calculates the change in the distance between the initial position and the current coordinates. Then, it may be determined as a parameter.

  Next, in step S104-1b, the left and right hand determination unit 134 determines the left and right hands from the moving direction of each contact information. The determination method at this time will be described with reference to FIG.

  FIG. 22 is a diagram illustrating a condition for determining the left and right hands based on whether any two points (point a and point b) in contact with the contact information detection unit 1 have moved in the X or Y axis direction. However, the coordinate system is the coordinate system shown in FIG. Normally, when the contact information detection unit 1 is installed at the mounting position as shown in FIG. 6, the point corresponding to the thumb is easy to move in the X-axis direction, and the point corresponding to the index finger is in the Y-axis direction. We focus on being easy to move. In the case of FIG. 21, any one of the two points that have transitioned from state a to state b is extracted. If the point A is extracted, the moving direction of the X coordinate calculated in step S103-4b becomes a negative direction. Next, the movement direction of the Y coordinate of the point B becomes the positive direction. These results are applied to the list of FIG.

  The determination condition of FIG. 22 is that when the right hand makes a transition from FIG. 5A to FIG. 5B, the X coordinate of one point moves in the negative direction, the Y coordinate of the other point moves in the positive direction, and the left hand Then, attention is paid to the fact that the X coordinate of one point moves in the positive direction and the Y coordinate of the other point moves in the positive direction. Therefore, the left and right hands can be determined by the combination of the moving directions of the two X and Y coordinates shown in FIG. That is, in the case of FIG. 21, the right hand is determined.

  As described above, the user moves the left and right hands by determining the left and right hands while performing a gesture in which the mode performs a function (in this example, changing the scale of the map by opening and closing fingers as shown in FIG. 5). There is no need to perform any kind of gesture just to make a decision, which is very convenient.

  In the case of FIGS. 21 and 22, the example has been described in which the left and right hands are determined based on the movement direction of the coordinates of the two points. However, the determination may be performed using the movement directions of the third, fourth, and fifth points. Here, an example in which determination is performed using five moving directions will be described.

  First, in step S103-1 b in FIG. 20, the gesture recognition unit 133 acquires a mode from the control target device 4. The mode acquired at this time is a mode in which the scale of the map can be changed as described above. The scale can be changed when the user enters the contact information detection unit 1 from the state in which all the fingertips of one hand are gathered and brought into contact with the contact information detection unit 1 and is opened while being brought into contact (or vice versa). To do. FIG. 23 illustrates the position of each coordinate acquired by the contact information detection unit 1 at that time, the movement of the user's hand, and part of the contact information acquired from the contact information history storage unit 132 in each state. Information. State a in FIG. 23A represents a state in which all fingertips of one hand are gathered, and state b in FIG. 23B represents a state in which all fingers of one hand are opened.

  Next, in steps S103-2b and S103-3b, the gesture recognizing unit 133 acquires the coordinates of each piece of contact information shown in FIG. 23, and the same number (in this case, five points) of contact information exists within a certain time. Determine whether. This process is the same as the process described above.

  Next, in step S103-4b, the movement direction of each coordinate is calculated. The calculation method is the same as that described above, and the result is shown in FIG.

  Next, in step S104-1b, the left and right hand determination unit 134 determines the left and right hands from the moving direction of each contact information. When the contact information detection unit 1 is installed at the attachment position as shown in FIG. 6, if a gesture as shown in FIG. 23 is performed, the point hitting the thumb when the state a is changed to the state b is We focus on the fact that it is easy to move in the X-axis direction and that the four points hitting other fingers are easy to move in the Y-axis direction. As a result, it is possible to determine the left and right hands by determining which of the moving directions shown in the list of FIG.

  In the case of FIG. 23, any one of the five points that have shifted from state a to state b is extracted. If the point A is extracted, the movement direction in the X-axis direction from the point A to the point A ′ is referred to. From the calculation result of FIG. 24, it can be seen that the moving direction of the X axis is a negative direction. Thereafter, the movement directions in the Y-axis direction of the other four points (point B, point C, point D, and point E) are referred to. Also from this calculation result of FIG. 24, it can be seen that the moving direction of the Y-axis is the positive direction with respect to all points. This applies to the condition shown in FIG. 25 in which the movement direction of the X coordinate is a negative direction and the movement direction of the Y coordinate is a positive direction, so the left / right hand determination unit 134 determines that the hand is a right hand.

  In addition, since the first point extracted this time is the point A, the left and right hands can be discriminated at a time. However, when the first extracted point is other than the point A, it does not correspond to the list of FIG. In such a case, the same process can be performed for all points to determine the left and right hands.

(4) When discriminating left and right hands from the moving direction + relative position (five fingers, thumb slide)
Another method for determining the left and right hands from the five points of contact information using the moving direction will be described below.

  In step S103-1 b in FIG. 20, the gesture recognition unit 133 acquires a mode from the control target device 4. The mode acquired at this time is a mode in which the scale of the map can be changed as described above. Then, the scale can be changed by moving the user to the left and right while only contacting the thumb from the state where all the fingertips of one hand are in contact with the contact information detection unit 1. FIG. 26 illustrates the position of each coordinate acquired by the contact information detection unit 1 at that time, the movement of the user's hand, and part of the contact information acquired from the contact information history storage unit 132 in each state. Information. In FIG. 26, state a represents a state in which all fingers of one hand are open, and state b represents a state in which only the thumb has been moved to the right from state a.

  Next, in steps S103-2b and S103-3b, the gesture recognizing unit 133 performs the same processing as the processing for the five points described above.

  Next, in step S103-4b, the gesture recognition unit 133 calculates the movement direction of each coordinate. The calculation method is the same as described above, and the result is shown in the list of FIG. In FIG. 27, “-” is described in the column of the X-axis movement direction and the Y-axis movement direction, which indicates that the point has not moved or the movement amount is very small. Since the gesture moves only the thumb, the other fingers do not move.

  Next, in step S104-1b, the left and right hand determination unit 134 determines the left and right hands from the movement direction and coordinate value of each coordinate. When the contact information detection unit is installed at the attachment position as shown in FIG. 6, when a gesture as shown in FIG. 26 is performed, the point corresponding to the thumb is X when the state a is changed to the state b. We focus on the ease of movement in the axial direction. Also, attention is paid to the fact that the Y coordinate of the point corresponding to the thumb is smaller than the Y coordinate of the other points. As a result, the left and right hands are determined when all the conditions are satisfied from the moving direction as shown in the list of FIG. 28 and the relative positional relationship of the contact coordinates of each fingertip.

  FIG. 27 shows the calculation result of each point and the moving direction in each axial direction. In the case of this example, the coordinate value of each point is used together with the moving direction. As the coordinate value, the coordinate value used at the start point or the end point when the movement direction is calculated is used. First, the left / right hand determining unit 134 extracts two points having the maximum and minimum X coordinates from the five past points (point A, point B, point C, point D, and point E) in the acquired coordinate history. . In this example, point A and point E are extracted. Next, out of the five most recent points, two points with the maximum and minimum X coordinates are extracted. In this example, point B 'and point E' are extracted.

  Next, it is assumed that the left and right hand determining unit 134 extracts an arbitrary point having the same ID and temporarily extracts a point A. At this time, it is determined whether it is included in the point extracted earlier. In the case of this example, since the X coordinate at the start point is minimum compared to other coordinates, the process proceeds to the next process. At this time, if not included, the same processing is performed for other points.

  Next, the left and right hand determination unit 134 compares the extracted one point with other Y coordinates, and determines whether the Y coordinate is minimum compared with the other four points. In the case of this example, it can be seen from FIG. Next, the left and right hand determining unit 134 refers to the X axis movement direction of the extracted point (point A). In this example, the movement direction of the point A in the X-axis direction is the positive direction, and as a result, all the conditions shown in FIG. 28 are satisfied, so the determination result of the left and right hands is the right hand. In this way, it is possible to determine the left and right hands from the contact information of five points.

(5) When discriminating left and right hands from the moving direction + moving amount (two fingers)
The flowchart of FIG. 29 is a process in the case where the user moves his / her finger in the state in which the finger is in contact with the contact information detection unit 1, calculates the moving direction and the moving amount, and uses these to determine the user's left and right hands. Shows the flow.

  First, in step S103-1c, the gesture recognition unit 133 acquires the current mode from the control target device 4. The mode in this example is a mode in which the scale of the map can be changed on the map screen, as in FIG. In order to execute this function, it is assumed that the scale can be changed by bringing the forefinger and the thumb into contact with the contact information detection unit 1 and opening and closing each other as shown in FIG.

  Next, in steps S103-2c to S103-5c, the gesture recognizing unit 133 performs the same processing as in steps S103-2b to S103-5b of FIG.

  Next, in step S104-1c, the left and right hand determining unit 134 calculates the amount of movement of each contact information from each contact information history. As a method for calculating the amount of movement, a general method for calculating the distance between two points (three square theorem) may be used, but other approximate values may be calculated in order to reduce the amount of calculation. And the value may be used.

  Next, in step S104-2c, the left and right hand determining unit 134 determines the left and right hands from the moving direction and moving amount of each contact information. This is based on the fact that when the index finger and thumb are used for opening and closing, the thumb is usually shorter than the index finger and the index finger is easier to move. For this reason, the amount of movement of the thumb is smaller than the amount of movement of the index finger. An example at this time is shown in FIG.

  In FIG. 30, when the transition from the state a to the state b is made, the pieces of contact information (point A to point A ′, point B to point B ′) move the lengths of l1 and l2 (l1 <l2), respectively. It shows how it is. In such a case, the left and right hand determining unit 134 has a finger corresponding to the point A (or point A ′) as a thumb and a point B (or point B ′) because the moving distance 11 is shorter than the moving distance l2. The corresponding finger is determined to be an index finger. And when a user's hand is an approach direction as shown in FIG. 6, this is determined to be operation with a right hand.

  If the left and right hands are determined by the method as described above, the left and right hands can be reliably determined even in the case of FIG.

(6) When discriminating the left and right hands from the moving direction + moving amount (3-point pinch operation)
The flowchart of FIG. 31 shows the relative movement direction and amount of movement of the two fingers and the other fingers in a state where the user touches the contact information detection unit 1 with two fingers and another finger. The flow of a process in the case of determining a user's left and right hand by calculating a position and using these is shown.

  First, in step S103-1d, the gesture recognition unit 133 acquires the current mode from the control target device 4. The mode in this example is a mode in which the scale of the map can be changed on the map screen, as in FIG. In order to execute this function, the scale can be changed by opening and closing each finger while the index finger and the thumb are in contact with the contact information detection unit 1 as shown in FIG. .

  Next, in the processing up to step S103-2d and step S103-3d, the gesture recognition unit 133 performs the same processing as in step S103-2b and step S103-2b.

  Next, in step S103-4d, the gesture recognition unit 133 calculates the movement amount of each contact information. This calculates the movement amount of each contact information by the method used in step S104-1c described above.

  Next, in step S103-5d, the gesture recognizing unit 133 calculates the movement direction of the top two pieces of contact information with the large movement amount. Here, in order to explain this processing, it will be explained using the example of FIG.

  FIG. 32 shows a state in which each piece of contact information (from point A to point A ′ and from point B to point B ′) is moving when the state a is changed to the state b. At this time, the movement distance from the point A to the point A ′ is l1, the movement distance from the point B to the point B ′ is l2, and the movement distance from the point C to the point C ′ is l3. It is assumed that> l1> l3. In such a case, in step S103-5d, the gesture recognizing unit 133 extracts points A (ID: 1) and B (ID: 2), which are points corresponding to l1 and l2, and determines their moving directions. calculate. At this time, the calculation method of the moving direction performs the same processing as the method shown in the result of FIG.

  Next, in step S103-6d, the gesture recognition unit 133 performs parameter calculation. This performs the same processing as step S103-5b of FIG.

  Next, in step S104-1d, the left and right hand determination unit 134 calculates the relative position of each piece of contact information. This relative relationship is calculated by paying attention to two points having a large moving distance and the X coordinates of other points. In the case of FIG. 32, paying attention to the X coordinates of two points (point A (ID: 1) and point B (ID: 2)) having a large moving distance and other points (point C, ID: 3), The X coordinate of C is compared with the X coordinates of point A and point B. In FIG. 32, the X coordinate of the point C is larger than the X coordinates of the point A and the point B.

  Next, in step S104-2d, the left and right hand determination unit 134 determines the left and right hands from the movement amount and relative position of each contact information.

  In the case of FIG. 32, the relative positional relationship between the X coordinates of two points (point A (ID: 1) and point B (ID: 2)) with a large amount of movement and other points (point C, ID: 3) is The point C having a small movement amount is larger than the points A and B having a large movement amount. In addition to opening and closing the thumb and index finger in contact with the contact information detection unit 1 as a gesture for changing the scale, other fingers are in contact with the contact information detection unit 1. Normally, when this operation is performed with the right hand, the other fingers are positioned on the right side (the X coordinate is large) from two points (thumb and index finger) having a large movement amount. When this operation is performed with the left hand, it is positioned on the left side (X coordinate is small). That is, in the case of FIG. 32, since the X coordinate of the point (point C) with a small movement amount is larger than the other, the left / right hand determining unit 134 determines that the right hand is present.

  By performing such processing, the left and right hands can be reliably determined even in the case of FIG. 18B that cannot be detected only in the moving direction.

(7) When discriminating the left and right hands from the relative position of the coordinates and the contact area The flowchart of FIG. 33 uses a specific one finger from a state in which the user makes two or more fingers contact the contact information detection unit. This shows the flow of processing when the left and right hands of the user are determined by tapping.

  In this example, the mode acquired from the control target device 4 is a mode in which a button is arranged on the screen and a specific function is executed by tapping the button. As a tapping method here, tapping with one finger (thumb) from a state where two or more fingers are in contact with the contact information detection unit 1 in advance. The flow of each process will be described using such an example.

  Note that a tap is a word representing an operation used when a button is selected using a touch screen, which is a pointing device used in a notebook computer or the like, and the method touches the touch screen for a short time. To release.

  First, in steps S103-1e and S103-2e, the gesture recognition unit 133 performs the same processing as steps S103-1a and S103-2a in FIG.

  Next, in step S103-3e, the gesture recognizing unit 133 determines whether at least one piece of the latest contact information among the contact information acquired from the contact information history storage unit 132 exists. Here, if no contact information exists, the process proceeds to step S101, and if present, the process proceeds to step S103-4e.

  Next, in step S103-4e, the gesture recognition unit 133 performs tap detection. A general method may be used as a tap detection method at this time. For example, a method of detecting a tap from an increase / decrease in the number of coordinates within a certain time using a history of acquired contact information can be considered. This example will be described with reference to FIG. When the number of coordinates of the contact information in the latest state (state c) in FIG. 34 (c) is four points, for example, when tap detection is performed during one second, the number of coordinates of the contact information is four (FIG. 34). (A) in state a), 5 (in state b in FIG. 34 (b)), 4 (in state c in FIG. 34 (c)), 1 from the number of current contact information. If it only increases, this is detected as a tap.

  Next, in step S103-5e, the gesture recognizing unit 133 proceeds to step S104-1e if tap detection is performed in step S103-4e, and proceeds to step S101 if not detected.

  Next, in step S104-1e, the left and right hand determining unit 134 refers to the contact area of each contact information at the time of tapping (timing at which the number of coordinates in contact increases). In the case of the example shown in FIG. 34, S1 ', S2', S3 ', S4', and S5 'are referred to as the contact area of each piece of contact information in the state b in FIG.

  Next, in step S104-2e, the left and right hand determining unit 134 compares the contact information that is the basis for detecting the tap with the contact area of the other contact information, and determines the contact information that is the basis for detecting the tap. It is determined whether the contact area is the largest. This focuses on the fact that when a human finger comes into contact with something, it is the thumb that has the largest contact area.

  In the case of the example shown in FIG. 34, when the state a is changed to the state c, the point A ′ (ID: 1) is caused by an increase or decrease in the number of coordinates. It is determined whether ') is larger than the contact area of other points (point B', point C ', point D', point E '). In the case of this example, the size of the contact area of each contact information in the state b is S1 '> S3'> S2 '> S4'> S5 ', so the process proceeds to step S104-3e. At this time, if the contact area of the contact information resulting from the tap detection is not the largest, the process proceeds to step S101.

  Next, in step S104-3e, the left and right hand determination unit 134 determines the left and right hands based on the relative positional relationship between the coordinate value of the contact information that is a trigger detected when the tap is detected and the coordinate value of the other contact information. This relative relationship indicates whether the X coordinate of the coordinate value of the contact information serving as a trigger is the largest or smallest compared to the coordinate values of the other contact information. This is usually a right hand when the detected contact area of the finger is larger than the other and the finger is on the left side of the other finger (position where the X coordinate is small), and the right side of the other finger (X coordinate is Focusing on the fact that it is in the left hand when in a large position.

  In the case of the example in FIG. 34, the appearance of the point A ′ (ID: 1) in the state b triggered the tap detection, so the point A ′ in the state b and the other points (point B ′, point C) Referring to the relative relationship of the coordinate value of the X coordinate with ', point D', point E '), point A' is the other point (point B ', point C', point D ', point E') In this case, it is determined to be the right hand because it takes the smallest value compared to the X coordinate.

  In step S104-2e, only the relative comparison of the contact areas is determined. However, the comparison may be performed after a certain threshold is provided. In step S104-3e, when calculating the relative positional relationship of coordinate values, only the X coordinate is used. Usually, the thumb has a smaller Y coordinate than other fingers, and therefore, when determining the left and right hands. The Y coordinate may be used together.

  In this way, by utilizing the characteristics of the thumb having a large contact area and the smallest (or largest) X-coordinate compared to other contact information, it is possible to determine the left and right hands together during a general operation of tapping. Therefore, it is not necessary to provide a separate mode for left / right hand determination. Further, it is not necessary for the user to perform a special operation in order to determine the left and right hands.

(8) When distinguishing left and right hands from the moving direction and contact area (two, coordinate conversion)
The flowchart of FIG. 35 shows the user's left and right hands by using the moving direction in which the two fingers are moved and the contact area of each contact information while the user is in contact with the contact information detecting unit 1. The flow of processing for determination is shown.

  First, in steps S103-1f to S103-5f, the gesture recognition unit 133 performs the same processing as the processing from steps 103-1b to S103-5b in FIG. The same applies to the acquisition mode and the function execution method.

  Next, in Step 104-1f and Step S104-2f, the left and right hand determination unit 134 extracts the contact area of each contact information, and constructs a left and right hand determination coordinate system. This process will be described with reference to FIG.

  FIG. 36A shows a state when the contact information detection unit 1 detects contact information of two points for the first time. At this time, the point A (ID: 1) is assumed to be larger than the contact area of the point B (ID: 2). Next, the coordinate value (x1, y1) of the contact information (point A) having the largest contact area among the contact information is set as the origin of the left and right hand determination coordinate system, and the coordinates (x1, y1) of the contact information having the large contact area are used. A vector component to the coordinates (x2, y2) of the contact information with a small contact area is calculated. Then, from the calculated vector component, the vector component rotated −45 ° with the clockwise direction around the origin (point A: O) as the positive direction is the X ′ axis, and the vector component rotated + 45 ° is the Y ′ axis. Define. In the case of FIG. 36 (a), the coordinate system shown in FIG. 36 (b) is constructed.

  Next, in step S104-3f and step S104-4f, the left and right hand determining unit 134 calculates the moving direction of each contact information in the left and right hand determining coordinate system, and determines the left and right hands from the moving direction. Normally, when the operation with the thumb and the index finger is performed while making contact with the contact information detection unit 1, the contact area of the thumb becomes larger, and further, the user performs an operation of opening and closing the thumb and the index finger. When it does, it pays attention to drawing the locus | trajectory near each other's finger from the movable range of the finger joint of the thumb and the index finger. In other words, the constructed left / right hand determination coordinate system calculates the moving direction using a coordinate system in which the vector component connecting two points is distributed at 45 ° with the thumb as the origin, and the user determines the left / right hand, thereby allowing the user to Regardless of the direction in which the hand enters the contact information detection unit 1, the left and right hands can be reliably determined.

  For example, in FIG. 36C, it is assumed that point A and point B have moved to point A 'and point B'. An X′Y ′ coordinate system is constructed as the left / right hand determination coordinate system. When the movement direction of each point at this time is calculated, the movement direction from the point A to the point A ′ is negative in the X′-axis direction ((x′1-0) <0) and Y′-axis direction. No movement ((y′1-0) = 0), the movement direction from the point B to the point B ′ is no movement in the X′-axis direction ((x′2′−x′2) = 0) It is assumed that the positive direction ((y′1′−y′2)> 0) in the Y′-axis direction. In such a case, the left and right hand determining unit 134 determines the right hand from the moving direction of each point according to FIG.

  By performing such processing, the left and right hands can be distinguished even when the contact information detection unit 1 is operated from an arbitrary direction.

  When calculating the movement direction, the movement amount of each component (X ′ coordinate, Y ′ coordinate) of each contact information is used as a movement direction parameter, so that the left and right hands can be determined with higher accuracy.

  As described above, the left and right hands may be determined by combining some methods shown in (1) to (8). Thereby, more reliable left and right hand determination becomes possible.

  As described above, since the input device and the input method according to the present invention can determine the left and right hands of the user using only the contact information detection unit, the application operates according to the left and right hands without increasing the cost. It can be used for an input device and an input method for a user to input a command or the like to a device such as a car navigation device.

The figure which shows an example of the hardware constitutions of the input device 100 in embodiment of this invention The figure for demonstrating the contact position detected with the touch panel which determines a contact position each independently by the X-axis and a Y-axis Functional block diagram of input device 100 in an embodiment of the present invention The flowchart which shows the flow of the whole process of the input device 100 in embodiment of this invention. It is a figure which shows an example of a user's operation input into the contact information detection part 1, (a) The figure which shows the state which closed the thumb and the index finger, (b) The figure which shows the state which opened the thumb and the index finger It is a figure which shows the example of the installation position to the vehicle of the contact information detection part 1, (a) A figure which shows an example at the time of being attached to the center console of a vehicle, (b) An example at the time of being attached to the handle of a vehicle The figure to show, (c) The figure which shows an example at the time of attaching to displays, such as a car navigation FIG. 5 is a diagram for explaining the operation of the contact information detection unit 1 in step S101 of FIG. 4, (a) a diagram illustrating an example of a state where an operator's hand is placed on the operation surface of the contact information detection unit 1; ) The contact point group detected by the contact information detection unit 1 is represented by a black circle group. Flowchart for explaining step S101 in FIG. 4 in detail The figure for demonstrating the calculation method of the contact coordinate and contact area of step S101-2 of FIG. 8, S101-3 The figure which shows an example of the position of the contact coordinate acquired by step S101-3 of FIG. Flowchart for explaining step S102 in FIG. 4 in detail It is a figure for demonstrating the information stored in the contact information log | history storage part 132, (a) The figure for demonstrating the state which the user contacted the contact information detection part 1 last time, (b) The contact information log | history storage part The figure which shows the information of each point stored in 132 FIG. 11 is a diagram for explaining the processing in step S102-4 in FIG. 11, (a) a diagram for explaining each previous contact information (5 points) and each contact information detected this time (5 points); The figure for demonstrating the contact information detected this time, (c) The figure which shows an example of the distance between each coordinate acquired last time (5 points) and this time (5 points) It is a figure for demonstrating the process of step S102-9 of FIG. 11, (a) The figure for demonstrating each last contact information (5 points) and each contact information (4 points) detected this time, (b). The figure which shows an example of the distance between each coordinate acquired last time (5 points) and this time (4 points) FIG. 11 is a diagram for explaining the processing in step S102-10 in FIG. 11; (a) a diagram for explaining each previous contact information (4 points) and each contact information detected this time (5 points); and (b). The figure which shows an example of the distance between each coordinate acquired last time (4 points) and this time (5 points) In steps S103 and S104 of FIG. 4, a flowchart showing a processing flow when the user's left and right hands are determined from the relative position of each coordinate of the contact information when the user touches the contact information detection unit 1 with a finger. FIG. 17 is a diagram for explaining a method for determining left and right hands in step S104-1a in FIG. 16; (a) a diagram for explaining a case where the user operates with the right hand; and (b) a case where the user operates with the left hand. Figure to do FIG. 17 is a diagram for explaining an example in which the determination of the left and right hands is impossible in step S104-1a in FIG. 16, (a) a diagram illustrating a case where the X coordinates of two points match or the distance between the X coordinates is equal to or less than a threshold value; b) A diagram showing a case where two points have the same Y coordinate or the distance between the Y coordinates is equal to or smaller than a threshold value. FIG. 17 is a diagram for explaining a method of determining the left and right hands from the relative positions of the coordinates of the five points of contact information in step S104-1a in FIG. 16, and (a) a diagram for explaining a case where the user operates with the right hand; (B) The figure for demonstrating the case where a user operates with the left hand In steps S103 and S104 of FIG. 4, a flowchart showing a processing flow in a case where the user's left and right hands are determined based on the moving direction in which the user moves his / her finger in a state where the user contacts the contact information detection unit 1. FIGS. 20A and 20B are diagrams for explaining a process of calculating a moving direction of each coordinate from each contact information history in step S103-4b of FIG. 20, (a) a diagram showing a state a with the thumb and index finger closed; and (b) a thumb. Showing state b with index finger open FIG. 20 is a diagram illustrating a condition for determining the left and right hands depending on which direction of the X and Y axes two arbitrary points (point a and point b) in contact with the contact information detection unit 1 are moved in step S104-1b in FIG. FIG. 20 is a diagram for explaining a case where the left and right hands of a user are determined using five moving directions in the flowchart of FIG. 20, (a) a diagram showing a case of a state a in which all fingertips of one hand are gathered; b) The figure which shows the case of the state b which opened all the fingers of one hand The figure which shows the calculation formula and moving direction of a moving direction in the case of determining a user's left and right hand using 5 moving directions in step S103-4b of FIG. The figure which shows the conditions which determine a user's left and right hand using 5 points | pieces of moving directions in FIG.20 S104-1b. In the flowchart of FIG. 20, it is a figure for demonstrating the method to determine a left and right hand by operation which moves only a thumb to the left and right, (a) The figure which shows the state a which has opened all the fingers of one hand, (b) State a The figure which shows the state b which moved only the thumb from right to right The figure which shows the moving direction of each point at the time of changing into the state b which moved only the thumb to the right from the state a which opened all the fingers of one hand of FIG. The figure which shows the conditions which determine a user's left and right hand using a moving direction in FIG.20 S104-1b when operation which moves only the thumb of FIG. 26 to right and left is carried out. In steps S103 and S104 of FIG. 4, the user moves his / her finger with the contact information detection unit 1 in contact, calculates the moving direction and moving amount, and uses these to determine the user's left and right hands. Flowchart showing the flow of processing The figure which shows the moving amount | distance of the thumb and the index finger at the time of opening and closing using the index finger and the thumb of FIG. In steps S103 and S104 of FIG. 4, the movement direction, movement amount, and other fingers of the two fingers moved while the user is in contact with the contact information detection unit 1 and two fingers. The flowchart which shows the flow of a process in the case of determining a user's left and right hand by calculating the relative position with and using these In the flowchart of FIG. 31, it is a figure for demonstrating the method to determine left and right hand, (a) The figure which shows the state a which closed the index finger and thumb, (b) The figure which shows the state b which opened the index finger and thumb, c) A diagram showing the moving distance of each point In Steps S103 and S104 in FIG. 4, the user's left and right hands are determined by tapping with one specific finger from a state where two or more fingers are in contact with the contact information detection unit. Showing the flow of processing The figure for demonstrating the method of determining a user's right and left hand by tapping using the one specific one finger from the state which the user of FIG. 33 made the contact information detection part contact two or more fingers. (A) A diagram showing a state a with four contact information coordinates, (b) a diagram showing a state b in which the number of contact information coordinates has changed from five to five, and (c) a contact from the state b. The figure which shows the state c which the number of coordinates of information changed to four In steps S103 and S104 in FIG. 4, the user moves the two fingers while the two fingers are in contact with the contact information detection unit 1 and uses the contact area of each piece of contact information. Flowchart showing the flow of processing when determining left and right hands 35 is a diagram for explaining a method of determining the left and right hands in the flowchart of FIG. 35, (a) a diagram showing a state when the contact information detection unit 1 detects two points of contact information for the first time, and (b) each contact information. The figure for demonstrating the left-right hand determination coordinate system constructed | assembled by extracting the contact area of (c), the figure which shows the movement state of each contact information in a left-right hand determination coordinate system, (d) left-right hand from the moving direction of each point The figure which shows the condition which judges

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Contact information detection part 2 Calculation part 3 Display part 4 Operation object apparatus 100 Input device 131 Related ID preparation part 132 Contact information log | history storage part 133 Gesture recognition part 134 Left and right hand discrimination | determination part 135 Command transmission part

Claims (8)

An input device for operating a device with a finger placed on an operation surface,
A contact information detection unit that detects a plurality of contact coordinates between the operation surface and a plurality of fingers;
A contact information history storage unit for storing contact coordinates detected by the contact information detection unit;
An associated ID creation unit that assigns an ID to each contact coordinate stored in the contact information history storage unit;
A gesture recognition unit for recognizing a gesture based on a history of each contact coordinate given the same ID by the related ID creation unit;
A left and right hand determination unit that determines whether a finger placed on the operation surface is a right hand or a left hand based on each contact coordinate stored in the contact information history storage unit;
An input device comprising: a command transmission unit configured to transmit a predetermined command to the device based on a recognition result by the gesture recognition unit and a determination result of the left and right hand determination unit. The input device according to claim 1, wherein the left and right hand determining unit determines the left and right hands based on a relative positional relationship between the plurality of contact coordinates. The contact information detection unit detects a contact area in addition to a plurality of contact coordinates between the operation surface and a plurality of fingers,
The contact information history storage unit stores contact coordinates and a contact area detected by the contact information detection unit,
The input device according to claim 2, wherein the left and right hand determination unit determines the left and right hands based on a relative positional relationship and a contact area of each contact coordinate stored in the contact information history storage unit. The gesture recognition means further calculates a moving direction of each contact coordinate based on a history of each contact coordinate given the same ID by the related ID creation unit,
The input device according to claim 1, wherein the left and right hand determination unit determines a left and right hand based on a movement direction of the plurality of contact coordinates calculated by the gesture recognition unit. The gesture recognition means further calculates a moving direction of each contact coordinate based on a history of each contact coordinate given the same ID by the related ID creation unit,
The left and right hand determination unit determines left and right hands based on a movement direction of the plurality of contact coordinates calculated by the gesture recognition unit and a relative positional relationship between the plurality of contact coordinates. Input device. The left and right hand determination unit further calculates a movement amount of the plurality of contact coordinates, and determines the left and right hands based on the calculated movement direction and movement amount of the plurality of contact coordinates. Input device. The contact information detection unit detects a contact area in addition to a plurality of contact coordinates between the operation surface and a plurality of fingers,
The contact information history storage unit stores contact coordinates and a contact area detected by the contact information detection unit,
The left and right hand determining unit determines a left and right hand based on a movement direction of the plurality of contact coordinates calculated by the gesture recognition unit and a contact area stored in the contact information history storage unit. The input device according to claim 4. An input method for operating a device with a finger placed on an operation surface,
A contact information detection step of detecting a plurality of contact coordinates between the operation surface and a plurality of fingers;
A contact information history storage step for storing the contact coordinates detected in the contact information detection step;
A related ID creating step of giving an ID to each contact coordinate stored in the contact information history storing step;
A gesture recognition step for recognizing a gesture based on the history of each contact coordinate given the same ID by the related ID creation step;
Left and right hand determination step for determining whether a finger placed on the operation surface is a right hand or a left hand based on each contact coordinate stored in the contact information history storage step;
An input method comprising: a command transmission step of transmitting a predetermined command to the device based on a recognition result in the gesture recognition step and a determination result in the left and right hand determination step.

Images