JP5605270B2 - Touch panel device, operation determination method, and operation determination program - Google Patents

Description

  The present invention relates to a touch panel device, an operation determination method, and an operation determination program.

  Touch panel devices provided in mobile phones, personal computers, etc. can intuitively operate movement, enlargement, reduction, etc. of objects by touching image data displayed on the screen and multiple points on the menu simultaneously. Provides a user interface. However, the conventional touch panel device has a small display and is configured on the premise of a screen operation when an individual (single user) uses it. Therefore, the conventional touch panel device does not have a function of identifying which finger is the same user when a plurality of people simultaneously operate a plurality of points on the screen. Therefore, when a plurality of people operate a large touch panel at the same time, the user cannot be identified.

  For example, when a meeting is performed in an office, a usage form in which a table having a touch panel display on the upper surface of the table and a plurality of people simultaneously operate meeting materials displayed on the display can be considered. However, the conventional touch panel device cannot identify such a multi-user / multi-touch method.

  Patent Document 1 discloses a touch panel type input device that identifies input from an operator located in a plurality of directions by identifying the direction of an input finger. Patent Document 2 discloses a touch panel device used by many people.

Japanese Patent Laid-Open No. 2008-3868 JP 2010-219785 A

  However, in the technique of Patent Document 1, the finger of the same person is identified from a plurality of fingers input by a plurality of persons using the direction of the fingers input on the touch panel. Therefore, a plurality of different fingers facing the same direction cannot be distinguished. Moreover, in the technique of patent document 2, when several people use simultaneously, each person cannot be identified.

  The present invention has been made in view of the above problems, and an object thereof is to provide a touch panel device, an operation determination method, and an operation determination program that can determine the operations of a plurality of users.

  In order to solve the above problems, a touch panel device disclosed in the specification includes a ridge detection unit that detects ridges of fingerprints of a plurality of fingers, and a ridge that connects a predetermined point of each ridge detected by the ridge detection unit. An area detection unit that detects a predetermined area in the direction of the line coupling line for each finger, and an operation determination unit that determines an operation of the plurality of fingers when a positional relationship between the areas detected by the area detection unit satisfies a predetermined condition Are provided.

  In order to solve the above problems, an operation determination method disclosed in the specification includes a ridge detection step for detecting fingerprint ridges of a plurality of fingers, and a predetermined point of each ridge detected in the ridge detection step. An area detection step for detecting a predetermined area in the direction of the ridge line to be connected for each finger, and the operation of the plurality of fingers is determined when a positional relationship between the areas detected in the area detection step satisfies a predetermined condition An operation determination step.

  In order to solve the above-described problem, an operation determination program disclosed in the specification includes, on a computer, a ridge detection step for detecting fingerprint ridges of a plurality of fingers, and each ridge detected in the ridge detection step. A region detecting step for detecting, for each finger, a predetermined region in a direction of a ridge connecting line connecting predetermined points, and the operation of the plurality of fingers when the positional relationship between the regions detected in the region detecting step satisfies a predetermined condition And an operation determination step for determining.

  According to the touch panel device, the operation determination method, and the operation determination program disclosed in the specification, operations of a plurality of users can be determined.

It is a schematic diagram for demonstrating the scene where several users use a touch panel. It is a figure for demonstrating operation of each user. It is a figure for demonstrating operation of each user. 1 is a block diagram of a touch panel device according to Embodiment 1. FIG. It is an example of fingerprint image data corrected by the image processing unit. It is a figure for demonstrating an example of a fingerprint data collection log. (A) is the figure which represented on the fingerprint image data the point from which the curvature of a fingerprint becomes maximum value, (b) is a figure for demonstrating an example of a ridge coupling | bonding management table. It is a figure showing a part of fingerprint. It is a figure for demonstrating a pattern. It is a figure for demonstrating a ridge coupling line. It is a figure for demonstrating extraction of the center area | region of a hand. It is a figure for demonstrating an example of the finger management table of the same hand. It is an example of the same finger management table. It is a figure for demonstrating the operation determination table and each operation. It is a flowchart for demonstrating an example of the process of the operation determination method of a touch panel. It is a figure for demonstrating a ridge coupling line. It is a figure for demonstrating correction | amendment of a ridge coupling line. It is a figure for demonstrating an example of the distance management table which records the relationship between fingerprint area and distance d. It is a flowchart showing the process of the operation determination method of the touchscreen in this modification. It is a figure for demonstrating the dry state of the same finger | toe. It is a flowchart of the process of the operation determination method of the touchscreen in this modification. It is a figure for demonstrating the angle which each finger | toe in the case of a touch panel operation makes. It is a figure for demonstrating the other example of a distance management table. It is a figure for demonstrating the case where a several user's finger | toe is arrange | positioned in the near position. It is a figure for demonstrating an example of the invalidation pattern table. 6 is a block diagram of a touch panel device according to Embodiment 2. FIG. It is a figure for demonstrating correction | amendment of the ridge joint line using joint information. It is a figure for demonstrating an example of the distance management table which records the relationship between the number of joints and distance d.

  First, usage scenes of a touch panel used by a plurality of users (multi-users) will be described. FIG. 1 is a schematic diagram for explaining a scene in which a plurality of users use a touch panel. Referring to FIG. 1, user A, user B, user C, and user D are having a meeting while displaying a proposal on the touch panel on the upper surface of the table. Assume that user A attempts to rotate the graphic contained in the proposal, user B attempts to scroll the screen, user C attempts to enlarge characters, and user D attempts to display a hyperlink. Further, it is assumed that the users A to D are performing each operation simultaneously.

The touch operation of each user is as follows.
1) User A tries to rotate the figure by drawing the arc with his right index finger with the right thumb as a fulcrum for the figure included in the proposal.
2) User B tries to scroll the proposal screen by sliding the right index finger and the middle right finger in the same direction.
3) User C tries to enlarge the character by pushing and spreading the right thumb and right index finger against the specific character string included in the proposal.
4) User D is about to display a hyperlink included in the proposal by clicking with a right index finger.

  In such a situation, it is difficult to determine each user's operation. In the example of FIG. 2, two users are each inputting two fingers to the touch panel. One user is trying to scroll the display screen by sliding the right index finger and the right middle finger in the same direction. The other user is trying to enlarge and display the right thumb and right index finger. However, if the former right hand middle finger and right hand index finger are close to the latter right hand index finger, the former right hand middle finger and right hand index finger and the latter right hand index finger may be determined as fingers input by the same user. In this case, the operation of each user cannot be determined.

  Thus, it is conceivable to identify the same user's finger from a plurality of fingers using the direction of the finger input to the touch panel. However, the fingers of a plurality of users may face the same direction. In the example of FIG. 3, one user tries to rotate the displayed figure by drawing the arc with the right hand index finger with the right thumb as a fulcrum. The other user tries to enlarge a specific character displayed on the touch display by pushing and spreading the right thumb and the right index finger. However, when the former right thumb and the latter right thumb and right index finger are pointing in the same direction, the operation of each user cannot be determined.

  In the method using information such as the input finger position and orientation, it is difficult to determine each user's operation. Therefore, in the following embodiments, a touch panel device, an operation determination method, and an operation determination program that can determine the operations of a plurality of users will be described.

  FIG. 4A is a functional block diagram of the touch panel device 100 according to the first embodiment. With reference to FIG. 4A, the touch panel device 100 includes a touch panel 10, an arithmetic processing unit 20, a data storage unit 30, and a power supply unit 40. The touch panel 10 includes a fingerprint information acquisition unit 11. The arithmetic processing unit 20 includes an image processing unit 21, a fingerprint data generation unit 22, a ridge combination processing unit 23, an identical hand determination unit 24, an identical finger determination unit 25, and an operation determination unit 26.

  FIG. 4B is a block diagram for explaining a hardware configuration of the touch panel device 100. Referring to FIG. 4B, the touch panel device 100 includes a CPU 101 in addition to the touch panel 10 and the data storage unit 30 in FIG. Each of these devices is connected by a bus or the like.

  A CPU (Central Processing Unit) 101 is a central processing unit. The CPU 101 includes one or more cores. The data storage unit 30 includes a RAM (Random Access Memory) 102, a ROM (Read Only Memory) 103, and the like. The RAM 102 is a volatile memory that temporarily stores programs executed by the CPU 101, data processed by the CPU 101, and the like. The ROM 103 is a nonvolatile storage device. The operation determination program according to the present embodiment is stored in the ROM 103. The nonvolatile storage device is not limited to the ROM, and may be a solid state drive (SSD) such as a flash memory or a hard disk driven by a hard disk drive.

  The operation determination program stored in the ROM 103 is expanded in the RAM 102 so as to be executable. The CPU 101 executes an operation determination program developed on the RAM 102. As a result, each function of FIG. 4A is realized. In addition, not only execution of a program but each function of Fig.4 (a) may be implement | achieved using each dedicated hardware.

  Referring to FIG. 4A again, the power supply unit 40 acquires power from the outside through an outlet or the like, and supplies power to the touch panel 10, the arithmetic processing unit 20, and the data storage unit 30. The data storage unit 30 stores data related to fingerprints, a table used for operation determination, and the like. Details of the data stored in the data storage unit 30 will be described later.

  The fingerprint information acquisition unit 11 is an optical sensor such as a fingerprint sensor. The fingerprint information acquisition unit 11 acquires fingerprint image data and fingerprint coordinate values of the user who is touching the touch panel 10 and passes the acquired fingerprint image data and fingerprint coordinate values to the image processing unit 21 in association with the fingerprint ID.

  The image processing unit 21 performs image correction processing such as noise removal on the fingerprint image data, and passes the corrected fingerprint image data and fingerprint coordinate values to the fingerprint data generation unit 22. FIG. 5 is an example of fingerprint image data corrected by the image processing unit 21. The fingerprint data generation unit 22 creates fingerprint data from the fingerprint image data and records it in the fingerprint data collection log together with the fingerprint coordinate values.

  FIG. 6 is a diagram for explaining an example of the fingerprint data collection log. Referring to FIG. 6, the fingerprint data collection log is stored in data storage unit 30 as a table. The fingerprint data collection log includes fingerprint data, fingerprint coordinates, hand center area, hand center ID, and update date and time in association with the fingerprint ID. In the initial state of the fingerprint data collection log, the center area of the hand and the center ID of the hand are not recorded. Details of the center region of the hand and the center ID of the hand will be described later.

  Fingerprint data includes feature point information (position, shape, etc. of feature points), ridge interval information, ridge direction information, coordinate value information of the fingerprint center, and a digitized dry state of the finger. Note that the data in the fingerprint data collection log may be automatically deleted after a predetermined time has elapsed. Thereby, the data in the fingerprint data collection log can be updated periodically.

  The ridge combination processing unit 23 extracts a point at which the curvature of the fingerprint has a maximum value from the fingerprint data and the corresponding coordinate value, and records it in the ridge combination management table. FIG. 7A is a diagram showing on the fingerprint image data the point at which the curvature of the fingerprint has a maximum value. FIG. 7B is a diagram for explaining an example of a ridge connection management table. The ridge connection management table is stored in the data storage unit 30. Referring to FIG. 7B, the ridge connection management table includes a point where the curvature of the fingerprint becomes a maximum value in association with the fingerprint ID.

  As shown in FIG. 7A, the ridge combination processing unit 23 obtains a ridge connection line by connecting points at which the curvature of each ridge becomes a maximum value. By connecting points at which the curvature becomes a maximum value, a direction substantially the same as the direction in which the finger extends can be acquired. By acquiring the direction in which the finger extends, the center direction of the hand can be acquired. The ridge combination processing unit 23 records the acquired center direction of the hand in the center direction management table of the hand.

  In addition, if the method of connecting the points at which the curvature becomes a maximum value is used, the center direction of the hand can be acquired even when only a part of the fingerprint can be collected as shown in FIG. Further, in the method of connecting the points at which the curvature becomes the maximum value, the center direction of the hand can be acquired for any of the patterns in FIGS. 9 (a) to 9 (e). In addition, the pattern of Fig.9 (a) is a vortex type. The pattern in FIG. 9B is a left flow type. The pattern in FIG. 9C is a right flow type. The pattern in FIG. 9D is a column shape. The pattern in FIG. 9 (e) has a wave shape.

  With reference to Fig.10 (a), the ridge coupling | bond process part 23 may acquire the direction from which the direction of the ridge which passes along the point where a curvature becomes the maximum value becomes a square shape as a center direction of a hand. That is, the same hand determination unit 24 may acquire, as the center direction of the hand, the direction in which the angle at which the ridge line at the point where the curvature becomes the maximum value is less than 180 degrees. In this case, the acquisition accuracy in the center direction of the hand is improved.

  FIG. 10B is a diagram for explaining an example of the hand center direction management table. The hand center direction management table includes the hand center direction (angle) in association with the fingerprint ID. The hand center direction management table is stored in the data storage unit 30. The center direction of the hand can be a rotation angle from the reference direction on the touch panel 10.

  The same hand determination unit 24 determines that each finger is a finger of the same hand when at least a part of the predetermined region in the direction of the ridge line obtained from the fingerprint of each finger overlaps each other. For example, the same hand determination unit 24 acquires the center region of the hand from the ridge line, and determines that each finger is a finger of the same hand when at least some of the center regions overlap each other.

  In addition, with reference to FIG. 11A, when the ridge coupling line obtained from the fingerprint of each finger is extended in the palm direction, each ridge coupling line does not necessarily converge to one point. Therefore, the same hand determination unit 24 extracts the center region of the hand using a predetermined distance d from the fingerprint position and the center direction of the hand recorded in the hand center direction management table. For example, referring to FIG. 11B, the same hand determination unit 24 determines the allowable range (x ± a, y ± b) from the point (x, y) moved from the fingerprint position toward the center of the hand by the distance d. ) As the center region of the hand. The position of the reference fingerprint can be arbitrarily selected and set, for example, from the point where the curvature becomes a maximum value. The same hand determination unit 24 records the obtained center region of the hand in the fingerprint data collection log of FIG.

  Further, the same hand determination unit 24 determines that fingerprint IDs in which at least a part of the center region of the hands overlap are fingers of the same hand, and sets the center IDs of the hands included in the fingerprint data collection log to the same value. The same hand determination unit 24 records the determination result in the finger management table of the same hand. FIG. 12 is a diagram for explaining an example of the finger management table of the same hand. The finger management table of the same hand includes a fingerprint ID in association with the management ID (hand center ID). The finger management table for the same hand is stored in the data storage unit 30. Note that the data in the finger management table of the same hand may be automatically deleted after a predetermined time has elapsed. Thereby, the data in the finger management table of the same hand can be updated periodically.

  By the way, when a multi-user simultaneously operates the touch panel 10 (multi-touch operation), it is necessary to identify which finger and which finger are the same finger in order to determine the operation method by operating the same finger. Therefore, the same finger determination unit 25 determines the same finger by comparing the fingerprint data recorded in the fingerprint data collection log with each other, and records the same finger in the same finger management table. FIG. 13 is an example of the same finger management table.

  The operation determination unit 26 performs the operation of the touch panel by comparing the operation determination table, the fingerprint data collection log in FIG. 6, the same finger management table in FIG. 12, and the same finger management table in FIG. 13. judge. FIG. 14A is an example of the operation determination table. Referring to FIG. 14A, the operation determination table indicates the detected index of the same hand, the time series change, and the operation as an operation number. In association with. FIG. 14B shows the operation No. It is a figure for demonstrating touch operation of 2 enlargement and reduction | decrease. FIG. 14C shows the operation No. It is a figure for demonstrating the touch operation of 5 slides. FIG. 14D shows an operation No. It is a figure for demonstrating the touch operation of 4 rotations. The operation determination table is stored in the data storage unit 30. The touch panel 10 executes an operation obtained as a result of determination by the operation determination unit 26.

  FIG. 15 is a flowchart for explaining an example of processing of a touch panel operation determination method. The flowchart of FIG. 15 is executed after fingerprint information is acquired by the fingerprint information acquisition unit 11 and fingerprint data is generated by the fingerprint data generation unit 22. Hereinafter, the flowchart of FIG. 15 will be described using specific data.

  First, the arithmetic processing unit 20 initializes (sets to 0) a value “X” that holds the number of fingers detected by the fingerprint information acquisition unit 11 (step S1). Next, the ridge connection processing unit 23 extracts a point where the curvature of the ridge reaches a maximum value, and records it in the ridge connection data management table of FIG. 7B (step S2). In the example of FIG. 7B, for the fingerprint ID assigned to the detected fingerprint “00000001”, there are 6 local maximum values ((10, 30), (10, 28), (10, 26), (10, 24), (10, 22), (10, 20)) are extracted. Next, with respect to the extracted six points, the ridge combination processing unit 23 sets the direction in which the shape of the ridge opens in a letter C as the center direction of the hand, and stores it in the center direction management table of the hand in FIG. Record. In the example of the hand center direction management table in FIG. 10B, it is recorded that the center direction of the hand whose fingerprint ID is “00000001” is 260 degrees.

  Next, the same hand determination unit 24 extracts the center of the hand whose fingerprint ID is “00000001” (step S3). It may be arbitrarily selected from the points where the corresponding fingerprint ID is the maximum value of the curvature of “00000001” and set as the start point, or the center position of the collected fingerprint data may be set. The same hand determination unit 24 assigns an allowable range to the center of the hand whose fingerprint ID is “00000001” and records it in the fingerprint data collection log of FIG. 6 as (0, 0) to (20, 20).

  Next, the arithmetic processing unit 20 substitutes “X + 1” for the value “X” (step S4). Next, the arithmetic processing unit 20 determines whether or not the value “X” is larger than the number of detected fingers (step S5). If “No” is determined in step S5, the process is executed again from step S2.

  If it is determined as “Yes” in step S5, the same hand determination unit 24 detects the center ID of the hand included in the fingerprint data collection log, and uses the same hand to identify a finger that overlaps at least a part of the center region of the hand. (Step S6). Next, in order to identify that the same finger has moved in a short time, the same hand determination unit 24 detects the same finger by comparing the fingerprint data included in the fingerprint data collection log with each other (step S7), The detection result is recorded in the same finger management table in FIG. In the example of FIG. 13, it is determined that the fingerprint IDs “00000001” and “00000007” are the same finger, and the fingerprint IDs “00000002” and “00000010” are the same finger.

  The operation determination unit 26 compares the finger management table of the same hand shown in FIG. 12, the same finger management table of FIG. 13, and the position information of each fingerprint ID included in the fingerprint data collection log of FIG. Judge the movement. Next, the operation determination unit 26 determines an operation with reference to the operation determination table of FIG. 14A (step S8). In the above example, the operation determination table management No. 1, the operation determination unit 26 determines whether the management No. It is determined that the operation No. 1 is being performed. Thereby, the touch panel 10 slides the screen.

  According to the present embodiment, even when a plurality of fingerprints at close positions are detected in a multi-user / multi-touch operation in which a plurality of users touch a touch panel with a plurality of fingers simultaneously, the fingers of the same hand can be identified. . Thereby, the operation of a plurality of users can be determined. In addition, since the same hand is determined using a predetermined region obtained from the ridge line, even when a plurality of fingers having the same orientation are detected, the fingers of the same hand can be identified. In addition, since it is not necessary to register a fingerprint in advance, it is possible to suppress a decrease in convenience.

  In the above example, the center area of the hand is acquired from the ridge line, but the present invention is not limited to this. Any region in the direction of the ridge line may be acquired. In the above example, each finger is determined to be a finger of the same hand when at least a part of the center region of the hand overlaps, but the present invention is not limited to this. Even if the predetermined regions in the direction of the ridge coupling line do not overlap with each other, it may be determined whether or not the plurality of fingers are fingers of the same hand according to the positional relationship. For example, when the predetermined points in the direction of the ridge line are within a predetermined distance, or when the predetermined areas in the direction of the ridge line are within a predetermined distance, the plurality of fingers are fingers of the same hand. You may determine that there is.

(Modification 1)
In addition, when connecting the point where the curvature of each ridge becomes the maximum value, a straight line may not be obtained. In that case, the ridge coupling processing unit 23 may correct the ridge coupling line. For example, referring to FIG. 16A and FIG. 16B, a point at which the curvature of the ridge reaches a maximum value may be shifted when connected to another point. Therefore, when the angle (θ) when connecting the three points is equal to or less than a preset reference value, the ridge connection processing unit 23 combines the next point without performing the connection processing with the corresponding point. May be.

  FIG. 16C is a diagram for explaining an example of a flowchart executed when the ridge joining processing unit 23 acquires a ridge joining line. Referring to FIG. 16C, the ridge combination processing unit 23 initializes (sets to 0) a value “X” that holds the number of ridges detected by the fingerprint data generation unit 22 (step S11). ). Next, the ridge combination processing unit 23 extracts an angle formed by a point at which the curvature reaches a maximum value in three adjacent ridges (step S12).

  Next, the ridge connection processing unit 23 determines whether or not the angle of the three points is equal to or less than the reference value θ (step S13). When it is determined as “No” in Step S13, the ridge joining processing unit 23 connects two points at the ends by skipping one point therebetween. Next, the ridge line combination processing unit 23 substitutes “X + 1” for the value “X” (step S15). If it is determined as “Yes” in step S13, three points are connected (step S16). Thereafter, step S15 is executed.

  After execution of step S15, the ridge combination processing unit 23 determines whether or not the value “X” is larger than the number of detected maximum ridge curvatures (step S17). If “No” is determined in step S17, the process is executed again from step S12. If it is determined as “Yes” in step S <b> 17, the execution of the flowchart ends. According to this modification, the accuracy in which the ridge connecting line is directed toward the center of the hand is improved.

(Modification 2)
It should be noted that the fingerprint center may be used when obtaining the ridge coupling line. FIG. 17A is a diagram for explaining the fingerprint center. Referring to FIG. 17A, the fingerprint center is a point indicating a semicircular flow included in the fingerprint, and may be called a core. In a vortexed fingerprint, the center of the vortex is the fingerprint center. The fingerprint center is used for alignment of the input image. FIGS. 17C to 17G show examples of fingerprint centers. The ridge combination processing unit 23 may correct the ridge combination line using the fingerprint center with reference to FIG. For this correction, a least square method or the like can be used. The ridge coupling line obtained by the least square method does not necessarily pass through the fingerprint center. By applying the fingerprint center information to the ridge combination process, the accuracy of extracting the center region of the hand can be improved, and the accuracy of finger identification of the same hand can be improved.

(Modification 3)
Since the fingerprint area is different for each finger, the fingerprint area may be used when determining the center region of the hand. For example, the fingerprint area of the thumb is larger than that of other fingers. The distance between the thumb and the center of the hand is larger than the distance between the other finger and the center of the hand. The center region of the hand may be obtained using this relationship. For example, the same hand determination unit 24 may change the distance d in FIG. 11B according to the fingerprint area.

  FIG. 18 is a diagram for explaining an example of a distance management table that records the relationship between the fingerprint area and the distance d in FIG. Referring to FIG. 18, the distance management table includes a fingerprint area and a distance d in association with the management ID. The distance management table can be set in advance. The method for calculating the fingerprint area is not particularly limited. According to this modification, since the distance d can be changed appropriately, the detection accuracy of the center region of the hand can be improved.

(Modification 4)
If the fingerprint area is relatively small, it can be determined that a finger other than the thumb has been detected. As an operation using a finger other than the thumb, scrolling or the like can be considered. In this case, it is conceivable that the fingers of the same hand are arranged at locations close to each other. Therefore, the process of extracting the center region of the hand may be omitted. For example, it is possible to use a characteristic in which the ridge intervals between fingers of the same hand are approximated. For example, in the case where the area of the collected fingerprint is relatively small, if a fingerprint whose ridge interval is close to the vicinity of the fingerprint (within a predetermined distance) is detected, it may be determined that the finger is the same hand. The ridge interval is recorded in the fingerprint data collection log in FIG.

  FIG. 19 is a flowchart illustrating a process of a touch panel operation determination method according to the present modification. The flowchart of FIG. 19 is executed after fingerprint information is acquired by the fingerprint information acquisition unit 11 and fingerprint data is generated by the fingerprint data generation unit 22.

  First, the arithmetic processing unit 20 initializes (sets to 0) a value “X” that holds the number of fingers detected by the fingerprint information acquisition unit 11 (step S21). Next, the same hand determination unit 24 determines whether or not the area of the fingerprint acquired by the fingerprint information acquisition unit 11 is greater than or equal to a threshold value (step S22). When it is determined as “Yes” in Step S22, each unit of the arithmetic processing unit 20 performs the same processing as Step S2 to Step S8 of FIG.

  When it is determined as “No” in step S22, the same hand determination unit 24 detects the finger related to the fingerprint that is detected within a predetermined distance from the fingerprint and whose ridge interval is approximate (the ridge interval difference is equal to or less than a predetermined value). Is determined to be the same finger as that of the fingerprint (step S23). Thereafter, the arithmetic processing unit 20 executes Step S8. According to this modification, the processing can be simplified.

(Modification 5)
In Modification 4, the ridge interval is used, but a dry state may be used. FIG. 20A to FIG. 20C are diagrams for explaining the dry state of the same finger. 20A shows a fingerprint image in a dry state, FIG. 20B shows a fingerprint image in a good state, and FIG. 20C shows a fingerprint image in a wet state. These dry states can be quantified.

  In general, the dry state of the fingers of the same hand approximates the same time period. By using this characteristic, processing efficiency can be improved. For example, in the case where the area of the collected fingerprint is relatively small, if a fingerprint having a dry state close to the fingerprint is detected, it may be determined that the finger is the same hand. The dry state is digitized and recorded in the fingerprint data collection log of FIG.

  FIG. 21 is a flowchart of the process of the touch panel operation determination method according to this modification. The flowchart of FIG. 21 is executed after fingerprint information is acquired by the fingerprint information acquisition unit 11 and fingerprint data is generated by the fingerprint data generation unit 22.

  First, the arithmetic processing unit 20 initializes (sets to 0) a value “X” that holds the number of fingers detected by the fingerprint information acquisition unit 11 (step S31). Next, the same hand determination unit 24 determines whether or not the area of the fingerprint acquired by the fingerprint information acquisition unit 11 is greater than or equal to a threshold value (step S32). When it is determined as “Yes” in Step S32, each unit of the arithmetic processing unit 20 performs the same processing as Step S2 to Step S8 of FIG.

  When it is determined as “No” in step S32, the same hand determination unit 24 detects the fingerprint related to the fingerprint that is detected within a predetermined distance and approximates the dry state (the numerical difference in the dry state is equal to or less than the predetermined value). Is determined to be the finger of the same hand as that of the fingerprint (step S33). Thereafter, the arithmetic processing unit 20 executes Step S8. According to this modification, the processing can be simplified.

(Modification 6)
The center area of the hand may be obtained using the angle of each finger. FIG. 22A to FIG. 22C are diagrams for explaining the angle formed by each finger during the touch panel operation. Referring to FIG. 22A, the angle formed by the thumb and the index finger when performing an enlargement or reduction operation is about 25 degrees as an example. Referring to FIG. 22B, an angle formed between the index finger and the middle finger when performing a scroll operation is, for example, about 0 degree. Referring to FIG. 22 (c), the angle formed by the thumb and forefinger when performing the rotation operation is about 45 degrees as an example.

  Thus, when the angle formed by the two fingers is close to 0 degrees, it can be estimated that the user is operating using the index finger and the middle finger. In this case, it is considered that the distance from the fingerprint of two fingers to the center region of the hand is substantially the same. On the other hand, when the angle formed by the two fingers is large, it can be assumed that the user is operating using the thumb and index finger. In this case, for a right-handed user, a thumb is arranged on the left side from the user toward the touch panel, and an index finger is arranged on the right side. By using the above points, the same hand determination unit 24 can change the distance d in FIG.

  FIG. 23 is a diagram for explaining another example of the distance management table. Referring to FIG. 23, the distance management table includes an angle and a distance d formed by two fingers in association with the management ID. The distance management table can be set in advance. For example, the relationship between the thumb and the index finger can be determined by setting the left side toward the touch panel from the user as the large angle side and the right side as the low angle side. The method for obtaining the angle between the two fingers is not particularly limited. For example, the angle between two fingers can be calculated from a fingerprint in the vicinity of the collected fingerprint, a fingerprint with a close ridge interval, and fingerprints with a close dry state.

(Modification 7)
When the fingers of different users are arranged close to each other, the operation of any user may be invalidated. FIG. 24 is a diagram for explaining a case where fingers of a plurality of users are arranged at close positions. Referring to FIG. 24, another user's finger may be placed between two fingers of the same hand. In this case, the operation determination unit 26 can suppress the operation contradiction and malfunction of the touch panel 10 by invalidating the operation with the other user's finger.

  FIG. 25 is a diagram for explaining an example of the invalidation pattern table. Referring to FIG. 25, the invalidation pattern table includes an invalidation pattern in association with the management ID. For example, when one fingerprint of a different hand is detected between two fingers of the same hand, the operation determination unit 26 invalidates the operation at a different point. In this case, an operation with two fingers of the same hand may be executed. Alternatively, when two different fingerprints of the same hand are detected between two fingers of the same hand, the operation determination unit 26 invalidates the operation with the two fingers sandwiching the two fingerprints. In this case, an operation with two fingers sandwiched between them may be executed.

  Note that whether or not the fingers are different hands may be determined according to whether the center regions of the hands are coincident, may be determined according to whether the ridge interval is approximate, or dry state You may determine according to whether or not. When invalidating, a combination pattern of angles formed by two fingers as shown in FIG. 24 may be used. In addition, when a fingerprint that matches the combination condition of the finger angle is detected in the vicinity of the fingerprint detected between the two fingers, the fingerprint angle detected between the finger and the finger angle detected in the vicinity is detected. You may invalidate the operation of the combination of fingers that match the combination pattern.

  FIG. 26A is a functional block diagram of the touch panel device 100a according to the second embodiment. The touch panel device 100a is different from the touch panel device 100 in FIG. 4A in that the touch panel 10 further includes a joint information acquisition unit 12. The joint information acquisition unit 12 is an infrared sensor or the like, and acquires the surface shape of the finger touching the touch panel 10. Thereby, the joint information acquisition part 12 can acquire the positional information on a joint. FIG. 26B is a block diagram for explaining a hardware configuration of the touch panel device 100a.

  FIG. 27A is a diagram for explaining an example of a finger surface shape acquired by the joint information acquisition unit 12. Referring to FIG. 27A, the finger joint is recognized as a line. Therefore, joint position information can be acquired by identifying this line with a different standard from the ridge. As an example, a line having a length greater than or equal to a predetermined value and an inclination equal to or less than a predetermined value can be acquired as a joint. The identification method is not particularly limited, and any method may be used.

  The joint information acquisition unit 12 records the joint region and the midpoint between the joint and the joint as joint position information in the joint region management table. FIG. 27B is a diagram for explaining an example of the joint region management table. Referring to FIG. 27B, the joint region management table includes a joint region and a joint midpoint in association with the management ID. The joint region management table is stored in the data storage unit 30.

  In this embodiment, the ridge connection processing unit 23 performs ridge connection using the joint position information information recorded in the joint region management table of FIG. For example, referring to FIG. 27C, the ridge connecting line may be corrected so as to pass through the joint region. If the ridge connecting line passes through the joint midpoint, the detection accuracy of the center region of the hand is improved. In the example of FIG. 27C, the ridge coupling line is corrected so as to pass through the joint middle point (5, 5) in the joint region of (1, 5) to (10, 5).

(Modification)
Referring to FIG. 28 (a), there is one thumb joint. There are two finger joints other than the thumb. On the other hand, the distance between the thumb and the center of the hand is larger than the distance between the other finger and the center of the hand. The center region of the hand may be obtained using this relationship. For example, the distance d in FIG. 11B may be changed according to the detected number of joints. FIG. 28B is a diagram for explaining an example of a distance management table that records the relationship between the number of joints and the distance d. Referring to FIG. 28B, the distance management table includes the number of joints and the distance d in association with the management ID. The distance management table can be set in advance. According to this modification, since the distance d can be changed appropriately, the detection accuracy of the center region of the hand can be improved. The number of joints can be the number detected within a predetermined distance from any part of the fingerprint toward the center of the hand.

  In each of the above examples, the fingerprint data generation unit 22 functions as a ridge detection unit that detects ridges of fingerprints of a plurality of fingers. In addition, the same hand determination unit 24 functions as a region detection unit that detects a predetermined region in the direction of a ridge connecting line connecting predetermined points of each ridge detected by the ridge detection unit for each finger. The operation determination unit 26 functions as an operation determination unit and an execution determination unit.

  Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and changes can be made within the scope of the gist of the present invention described in the claims. It can be changed.

(Appendix)
(Appendix 1)
A ridge detector for detecting fingerprint ridges;
A region detecting unit for detecting a predetermined region in a direction of a ridge connecting line connecting predetermined points of each ridge detected by the ridge detecting unit with respect to a plurality of fingers;
An operation determination unit that determines an operation of the plurality of fingers when a positional relationship of regions detected by the region detection unit satisfies a predetermined condition.
(Appendix 2)
The touch panel device according to appendix 1, wherein the operation determination unit determines an operation of the plurality of fingers when at least a part of the plurality of regions detected by the region detection unit overlaps.
(Appendix 3)
The touch panel device according to appendix 1 or 2, wherein the ridge connecting line is a line that connects points at which the curvature becomes maximum in each ridge.
(Appendix 4)
The predetermined area detected by the area detection unit is an area including a point at a predetermined distance from any part of the fingerprint in the direction of the ridge line. The touch panel device described.
(Appendix 5)
The touch panel device according to appendix 4, wherein the region detection unit changes the predetermined distance according to an area of a fingerprint.
(Appendix 6)
The touch panel device according to appendix 4, wherein the region detection unit changes the predetermined distance by using an angular difference between ridge bond lines obtained from two fingers.
(Appendix 7)
The region detection unit changes the predetermined distance according to the number of finger joints within a predetermined value in the direction of the ridge coupling line, where the distance from any part of the fingerprint is within a predetermined value. 4. The touch panel device according to 4.
(Appendix 8)
The touch panel device according to any one of appendices 1 to 7, wherein the region detection unit detects the predetermined region after correcting the ridge line using a fingerprint center.
(Appendix 9)
The touch panel device according to any one of appendices 1 to 8, wherein the region detection unit detects the predetermined region after correcting the ridge line so as to pass through a joint of a finger.
(Appendix 10)
When the area of the fingerprint is less than or equal to a predetermined value, the region detection unit skips the process of detecting the predetermined region, and the operation determination unit determines that the difference in ridge spacing among a plurality of fingers within a predetermined distance is a predetermined value. The touch panel device according to any one of appendices 1 to 9, wherein operations of a plurality of fingers are determined as follows.
(Appendix 11)
When the area of the fingerprint is less than or equal to a predetermined value, the region detection unit omits the process of detecting the predetermined region, and the operation determination unit determines the difference in the dry state quantified among a plurality of fingers within a predetermined distance. 10. The touch panel device according to any one of appendices 1 to 9, wherein the operation of a plurality of fingers with a predetermined value or less is determined.
(Appendix 12)
An execution determination unit that determines whether or not to perform the operation determined by the operation determination unit;
When the predetermined region detected by the region detecting unit satisfies a predetermined condition, a finger whose predetermined region detected by the region detecting unit does not satisfy the predetermined condition is sandwiched between the two fingers satisfying the predetermined region detected by the region detecting unit Furthermore, it determines with not performing operation of the finger pinched | interposed between the said 2 fingers, The touchscreen apparatus in any one of the additional notes 1-11 characterized by the above-mentioned.
(Appendix 13)
An execution determination unit that determines whether or not to perform the operation determined by the operation determination unit;
The implementation determining unit includes a finger having a difference between a ridge line interval of the two fingers equal to or greater than a predetermined value between two fingers satisfying a predetermined region detected by the region detecting unit. When it exists, it determines with not performing operation of the finger pinched | interposed between the said 2 fingers, The touchscreen apparatus in any one of the additional notes 1-11 characterized by the above-mentioned.
(Appendix 14)
An execution determination unit that determines whether or not to perform the operation determined by the operation determination unit;
In the execution determination unit, a finger whose difference from the dry state of the two fingers is equal to or greater than a predetermined value is disposed between two fingers satisfying a predetermined region detected by the region detection unit. The touch panel device according to any one of appendices 1 to 11, wherein it is determined that an operation of a finger sandwiched between the two fingers is not performed.
(Appendix 15)
An execution determination unit that determines whether or not to perform the operation determined by the operation determination unit;
The execution determination unit determines that the operation determined by the operation determination unit is not performed when the angle difference between the ridge bond lines obtained from two fingers is not within a predetermined range. The touch panel device according to any one of 11.
(Appendix 16)
An execution determination unit that determines whether or not to perform the operation determined by the operation determination unit;
When the fingerprint matching the combination condition of the finger angle is detected in the vicinity of the fingerprint detected between the two fingers, the execution determining unit is detected in the vicinity of the fingerprint detected between the fingers. The touch panel device according to any one of appendices 1 to 11, wherein it is determined that an operation of a combination of fingers that matches a combination pattern of finger angles is not performed.
(Appendix 17)
A ridge detection step for detecting fingerprint ridges;
A region detecting step of detecting a predetermined region in the direction of a ridge connecting line connecting predetermined points of each ridge detected in the ridge detecting step with respect to a plurality of fingers;
An operation determination step for determining an operation of the plurality of fingers when the positional relationship of the areas detected in the area detection step satisfies a predetermined condition.
(Appendix 18)
On the computer,
A ridge detection step for detecting fingerprint ridges;
A region detecting step of detecting a predetermined region in the direction of a ridge connecting line connecting predetermined points of each ridge detected in the ridge detecting step with respect to a plurality of fingers;
An operation determination program for executing an operation determination step of determining the operations of the plurality of fingers when the positional relationship of the areas detected in the area detection step satisfies a predetermined condition.

DESCRIPTION OF SYMBOLS 10 Touch panel 11 Fingerprint information acquisition part 12 Joint information acquisition part 20 Arithmetic processing part 21 Image processing part 22 Fingerprint data generation part 23 Ridge joint process part 24 Same hand determination part 25 Same finger determination part 26 Operation determination part 30 Data storage part 40 Power supply unit 100 Touch panel device

Claims (10)

A ridge detector for detecting fingerprint ridges;
A region detecting unit for detecting a predetermined region in a direction of a ridge connecting line connecting predetermined points of each ridge detected by the ridge detecting unit with respect to a plurality of fingers;
An operation determination unit that determines an operation of the plurality of fingers when a positional relationship of regions detected by the region detection unit satisfies a predetermined condition.   The touch panel device according to claim 1, wherein the operation determination unit determines an operation of the plurality of fingers when at least a part of the plurality of regions detected by the region detection unit overlaps.   The touch panel device according to claim 1, wherein the ridge connecting line is a line that connects points at which the curvature becomes maximum in each ridge.   The predetermined region detected by the region detection unit is a region including a point at a predetermined distance from any part of the fingerprint in the direction of the ridge coupling line. The touch panel device according to 1.   The touch panel device according to claim 4, wherein the region detection unit changes the predetermined distance according to an area of a fingerprint.   5. The touch panel device according to claim 4, wherein the region detection unit changes the predetermined distance by using an angular difference between ridge coupling lines obtained from two fingers.   The region detection unit changes the predetermined distance according to the number of finger joints within a predetermined value in a distance from any part of the fingerprint in the direction of the ridge coupling line. Item 5. The touch panel device according to Item 4.   The touch panel device according to claim 1, wherein the region detection unit detects the predetermined region after correcting the ridge coupling line using a fingerprint center. A ridge detection step for detecting fingerprint ridges;
A region detecting step of detecting a predetermined region in the direction of a ridge connecting line connecting predetermined points of each ridge detected in the ridge detecting step with respect to a plurality of fingers;
An operation determination step for determining an operation of the plurality of fingers when the positional relationship of the areas detected in the area detection step satisfies a predetermined condition. On the computer,
A ridge detection step for detecting fingerprint ridges;
A region detecting step of detecting a predetermined region in the direction of a ridge connecting line connecting predetermined points of each ridge detected in the ridge detecting step with respect to a plurality of fingers;
An operation determination program for executing an operation determination step of determining the operations of the plurality of fingers when the positional relationship of the areas detected in the area detection step satisfies a predetermined condition.

Images