CN1698279A

CN1698279A - Pointing device having fingerprint image recognition function, fingerprint image recognition and pointing method, and method for providing portable terminal service using thereof

Info

Publication number: CN1698279A
Application number: CNA2004800006623A
Authority: CN
Inventors: 朱圣哲
Original assignee: Mo Bisuo
Current assignee: Mo Bisuo; Mobisol Inc
Priority date: 2003-06-30
Filing date: 2004-06-30
Publication date: 2005-11-16
Also published as: KR20050002463A; KR100553961B1

Abstract

A pointing device having a fingerprint recognition function, a fingerprint image recognition and pointing method, and a method for providing a portable terminal service using the same are disclosed. In the pointing device having a fingerprint recognition function and a fingerprint recognition method, fingerprint images having small sizes are mapped to generate a lare fingerprint image or a small fingerprint image extracted from the large fingerprint image so that the pointing device performs user recognition and pointer control. As a result, respective sensors for the user recognition and for the pointer control are not comprised in the pointing device, but only one kind of sensors for performing both functions of user recognition and pointer control is comprised in the pointing device according to an embodiment of the present invention. Also, it is possible to easily embody miniaturization of a portable terminal device because the user recognition wich requires a large fingerprint can be performed only with a small fingerprint image, thereby reducing manufacturing cost. Additionally, important information in the portable terminal device having the pointing device can be protected by selectively limiting kinds of service usable in the portable terminal device depending on user recognition.

Description

Have pointing device, fingerprint image identification and the fix-point method of fingerprint image recognition function and use them to be used to provide the method for portable terminal service

Technical field

The present invention relates generally to a kind of pointing device and fingerprint identification method thereof with fingerprint image recognition function, more particularly, relate to a kind of by using a kind of transducer to carry out User Recognition and point control and do not use the pointing device and the fingerprint image recognition methods thereof of user identification sensor and point control transducer.

Background technology

Generally speaking, be meant widely used XY tablet, tracking ball and mouse in desktop computer, perhaps refer to widely used touch panel or touch pad in portable terminal such as laptop computer as the pointer of pointing device.Recently, make the optical mouse of using up obtain using.

Attempt the effort that biostatistics is integrated in electronic equipment and communication apparatus and the ancillary equipment thereof has been improved recently.The notable attribute of biostatistics is to alleviate the loss that under any circumstance causes owing to extraneous factor, the trouble of stealing, forgeing or reproduce, and this feature is best.When using this feature, whom the function of auditing fully followed the trail of and invaded safety.Specifically, use the user recognition technology of fingerprint to obtain positive commercialization, and visit and support this user recognition technology to be easy to, because it discerns the user by end user's feature.As a result, various researchs and carried out very big development in this field have been carried out.

Recently, developed the technology that the User Recognition that will use fingerprint is introduced pointing device.In pointing device, the fingerprint that inner fingerprint identification device is discerned fingerprint and the fingerprint that will before register and identification by predetermined window from finger surface compares, and when comparative result is independent of the fixed point function and is identical, confirms this fingerprint.Fig. 1 shows the fingerprint recognition optical mouse as example.As shown in Figure 1, fingerprint recognition optical mouse 1 has and general identical shape and the identical functions of mouse, but comprises fingerprint recognition window 2 in the part that right thumb touches.If right thumb touches fingerprint recognition window 2, then inner fingerprint Identification sensor (not shown) is discerned thumbprint and the fingerprint of identification and the fingerprint of previous registration is compared, with definite identification to the user.

In traditional fingerprint recognition, obtain the fingerprint of minimum required in the User Recognition.That is to say, in the fingerprint recognition optical mouse situation of Fig. 1, obtain the fingerprint of about 100 * 100 pixels, be used for fingerprint recognition.Now, on market, can buy the optical mouse of the fingerprint image that is used for once surveying 96 * 96 pixels.

In addition, fingerprint recognition can be introduced in the pointing device that uses fingerprint.In other words, in current pointing device, provide the technology of using fingerprint and using fingerprint recognition user's pointer simultaneously of controlling.Yet, in the prior art, be used for fingerprint recognition and obtain transducer with the fingerprint that obtains bigger fingerprint image and be used for User Recognition and be used for steering needle and obtain transducer with the fingerprint that obtains less fingerprint image and be included in a pointing device.

Fig. 2 shows and comprises that two fingerprints obtain the portable terminal of transducer.Fig. 2 (a) shows the part of portable computer (laptop computer) and the part that Fig. 2 (b) shows PDA.The fingerprint that has comprised the identification user in the situation of Fig. 2 (a) obtains transducer 3 with the fingerprint that is used for fingerprint recognition of the fingerprint that confirms this user and is used to control the pointer of pointer that is presented at the monitor of laptop computer with finger controls transducer 4.Yet the indicating needle controller in laptop computer is not to use fingerprint recognition and is to use pressure or the capacitance variations that causes of stylus by finger.In addition, in the situation of Fig. 2 (b), comprised respectively and be used for the User Recognition fingerprint and obtain transducer 5 and indicating needle controller transducer 6.Usually, in the situation that is used for controlling the laptop computer that moves of the cursor on the monitor according to the mobile message of finger, the fingerprint image of about 20 * 20 pixels is enough to obtain be used for the mobile message of the fingerprint of point control, but, for User Recognition, requiring has the fingerprint image that surpasses about 100 * 100 pixels.In the situation of the laptop computer of reality, the fingerprint that can buy with about 5 * 5mm obtains the product that transducer obtains the view data of about 100 * 100 pixels.

Though it is high-speed to embody small-power by the small data that is used for point control that uses about 20 * 20 pixels,, need obtain and analyze the big fingerprint image that is used for fingerprint recognition of 100 * 100 pixels.As a result, the fingerprint that need be used for User Recognition obtains

transducer

3 and 5 and pointer control transducer 4 and 6.Because these two kinds of fingerprints obtain transducer and are included in the pointing device,, the appearance of this pointing device sees and can not solve the technical complexity that these two kinds of fingerprints of driving obtain transducer so looking ill.Therefore, two kinds of fingerprints have been installed have in the prior art been obtained transducer, had a negative impact for the miniaturization that makes electronic device and device attenuation and become simple.

Therefore, need a kind ofly be used for obtaining transducer by the fingerprint that is used for User Recognition that uses portable terminal carries out User Recognition and carries out the method for point control by obtaining a plurality of little fingerprint images simultaneously.

Description of drawings

Fig. 1 is the perspective view that is used for traditional optical mouse of fingerprint recognition.

Fig. 2 shows the view of example of portable terminal that comprises the conventional fingerprint transducer that is used for fingerprint recognition and be used for traditional navigation panel of pointer control.

Fig. 3 shows the view according to the structure of the pointing device of the first embodiment of the present invention.

Fig. 4 shows the view according to the processing of displacement calculating data of the present invention.

Fig. 5 shows the view according to the processing of mapping fingerprint image of the present invention.

Fig. 6 shows the view of the structure of pointing device according to a second embodiment of the present invention.

Fig. 7 shows the view of mapping from the processing of the fingerprint image of a plurality of fingerprint deriving means acquisitions shown in Figure 6.

Fig. 8 show according to of the present invention first or the pointing device of second embodiment in the flow chart handled of fingerprint recognition.

Fig. 9 shows the detail flowchart of the processing of mapping fingerprint image in the virtual image space in Fig. 8.

Figure 10 shows the flow chart according to the processing of the pointer in the control pointing device of the present invention.

Figure 11 shows the view of structure of the pointing device of a third embodiment in accordance with the invention.

Figure 12 shows 3: 1 of being installed in the trickle space according to the present invention and contracts and resemble the view of Design for optical system example.

Figure 13 shows according to of the present invention contracting by application and resembles optical system and obtain the view of the example of the fingerprint image that device obtains from fingerprint.

Figure 14 shows the view of the pointing device of a fourth embodiment in accordance with the invention.

Figure 15 shows a kind of view of method that is used for extracting from the fingerprint image of M * N pixel the fingerprint image of m * n pixel.

Figure 16 show according to the of the present invention the 3rd or the 4th embodiment be used to the flow chart carrying out User Recognition and carry out the method for fixed point control simultaneously.

Figure 17 shows the view of the structure of pointing device according to a fifth embodiment of the invention.

Figure 18 shows the flow chart of the operation of pointing device according to a fifth embodiment of the invention.

Figure 19 showed according to a kind of being used for by using fingerprint identification function to limit the flow chart of method of the use of the mobile terminals equipment that depends on the user of the present invention.

Figure 20 shows the view according to the example of the portable terminal that comprises pointing device of the present invention.

Summary of the invention

Technical theme

The objective of the invention is to improve fingerprint identification method, thereby only use a kind of transducer just can carry out the control of User Recognition and pointer simultaneously, and do not need to be used for User Recognition and to be used for each fingerprint Identification sensor that pointer is controlled.

Technical solution

In one embodiment, a kind of pointing device with fingerprint image recognition function comprises: at least one or a plurality of fingerprint deriving means are used for obtaining according to predetermined period the fingerprint image of finger surface; The feature point extraction device is used for extracting one or more at least characteristic points from the fingerprint image that obtains; Mobile sniffer is used to calculate the displacement data between the characteristic point of fingerprint image of extraction, to survey the mobile message of fingerprint image; Mapping device is used for shining upon fingerprint image according to mobile message in inner virtual image space; Recognition device when the overall dimensions of fingerprint image of mapping reaches previous when size is set, compares the fingerprint image of the whole mapping fingerprint image with previous registration, and determines the identification to fingerprint; And arithmetic unit, be used to receive from the displacement data of mobile sniffer with displacement data and calculate pointer the direction and the distance that move.

In one embodiment, a kind of pointing device with fingerprint identification function comprises: fingerprint deriving means (first operation cycle), be used to obtain the fingerprint image of finger surface, and obtain steering needle by once two dimensional image only; Fingerprint identification unit (second operation cycle), the characteristic point that is used for the fingerprint image that will obtain compares with the characteristic point of the fingerprint image of previous registration, the user of the fingerprint image that obtains with identification; And fixed point control unit (the 3rd operation cycle), be used for according to depending on that the partial data of the image that first operation cycle obtained surveys mobile message, with the displacement data that comes the calculated fingerprint image according to mobile message, to calculate the moving direction and the distance of pointer.

In one embodiment, a kind of method that is used to discern the fingerprint that User Recognition uses may further comprise the steps: use predetermined fingerprint to obtain transducer and obtain at least one or a plurality of fingerprint image according to the cycle that is provided with; From the fingerprint image that obtains, extract one or more at least characteristic points; Mapping first fingerprint image in the ad-hoc location in virtual image space; Displacement data between the characteristic point of second fingerprint image that calculates the characteristic point of first fingerprint image and obtained in the following one-period after the cycle of obtaining first fingerprint image; In virtual image space, shine upon second fingerprint image with displacement data; And when the overall dimensions of the fingerprint image that shines upon reaches the size of previous setting, the characteristic point of the fingerprint image of the whole mapping characteristic point with the fingerprint image of previous registration is compared, and determine identification to fingerprint in virtual image space.

In one embodiment, a kind of fix-point method of pointer control appliance is provided, this pointer control appliance has size than the also little imageing sensor of preliminary dimension required in fingerprint recognition, this method may further comprise the steps: use the predetermined fingerprint of the removable pointer of control to obtain transducer, obtain at least one or a plurality of fingerprint image of M * N pixel according to first operation cycle on the finger surface; By according to second operation cycle extract minutiae and the characteristic point the extracted characteristic point with the fingerprint image of previous registration compared from the fingerprint image that obtains, determine identification to the user of fingerprint image; From the fingerprint image that obtains, extract the fingerprint image of m * n pixel according to the 3rd operation cycle; The displacement data of the fingerprint image by calculating the m * n pixel extracted is surveyed the mobile message of each fingerprint image; And use displacement data calculating and output pointer with the direction and the distance that move.

In one embodiment, a kind of pointing device with fingerprint image recognition function comprises: at least one or a plurality of fingerprint deriving means are used for obtaining according to predetermined period or requirement once in a while the image of finger surface; Mobile sniffer is used to calculate the displacement data of the fingerprint image of acquisition, to survey the mobile message of each fingerprint image; Arithmetic unit is used to receive the displacement data from mobile sniffer, uses displacement data to calculate pointer with the direction and the distance that move; Storage device is used to shine upon from fingerprint deriving means and the fingerprint image of arithmetic unit acquisition and the displacement data of fingerprint image; CPU is used to analyze and handle the data of arithmetic unit and image storage space.

Embodiment

With reference to the accompanying drawings the present invention is described in detail.

The pointing device of Fig. 3 comprises light-emitting device 22, beam condensing unit 23, fingerprint deriving means 24, feature point extraction device 25, storage device 26, mobile sniffer 27, mapping device 28, virtual image space 29, recognition device 30 and arithmetic unit 31.

With reference to figure 3, when the finger 20 that will obtain fingerprint image touched Transparent Parts 21, light-emitting device 22 was at the surface light emitting as the finger 20 that touches object.Light-emitting device 22 comprises at least one or a plurality of light-emitting diode.

Beam condensing unit 23 is assembled the fingerprint image of launching to contact object by from light-emitting device 22 that light produced.Can make with the light convex lens as beam condensing unit 23.

Fingerprint deriving means 24 is surveyed the simulation fingerprint image of being assembled by beam condensing unit 23 and will be simulated fingerprint image and is converted to the digital finger-print image.Fingerprint deriving means 24 comprises optic sensor array, two-dimensional arrangements a plurality of cmos image sensors (being abbreviated as " CIS ").Here, obtain a plurality of fingerprint images in the cycle that fingerprint deriving means 24 formerly is provided with.Fingerprint deriving means 24 manufactures and is suitable for the small portable terminal equipment, thereby obtains little fingerprint image.For example, be applicable to that the microsensor that is used to obtain less than about 20 * 20 pixels that fixed point is controlled is used as fingerprint deriving means 24.Here, the known equipment of those skilled in the art can be used as light-emitting device 22, Transparent Parts 21, beam condensing unit 23 and fingerprint deriving means 24.

The lamp that penetrates from light-emitting device 22 is mapped to the surface of finger 20 and reflects according to the figure on the surface of finger 20.Fingerprint deriving means 24, form one by beam condensing unit 23 from the light of pointing the reflection of 20 basal surface and resemble (phase).Be formed on resembling in the fingerprint deriving means 24 and be converted to the digital finger-print image by fingerprint deriving means 24.Obtaining on time shaft of fingerprint image carried out apace continuously.

Feature point extraction device 25 extracts one or more at least characteristic points from each fingerprint image that is obtained by fingerprint deriving means 24 in predetermined period.These characteristic points comprise the lines length and the direction of fingerprint image and lines separates or the position data of termination.

Fingerprint image that storage device 26 storage is obtained from fingerprint deriving means 24 and the information of extracting from feature point extraction device 25 about characteristic point.

The characteristic point of the fingerprint image of mobile sniffer 27 from be stored in storage device 26 is surveyed the mobile degree of each fingerprint image.At this, mobile sniffer 27 is by using mobile estimating method, surveying the mobile degree of fingerprint according to the displacement data (direction and distance) of the mobile computing characteristic point of fingerprint.In this way, mobile sniffer 27 is surveyed the mobile degree of fingerprint image by the characteristic point of the fingerprint image that obtains in the cycle that more formerly is provided with.Obtaining of the mobile message of the fingerprint image in the fingerprint recognition and feature point extraction and fingerprint image is important factor, because how the reliability mobile and fingerprint recognition of fingerprint image is extracted to distinguish according to characteristic point reliably comes.

Mapping device 28 receives the displacement data (direction and distance) of the characteristic point of fingerprint image, it is the mobile message that moves that depends on fingerprint image from mobile sniffer 27, and mapping device 28 uses displacement data to determine the position that mobile fingerprint image will shine upon in virtual image space 29.Next, mapping device 28 shines upon each fingerprint image according to the position of determining.When the mapped device 28 of fingerprint image shone upon, formerly in the middle of the characteristic point of obtaining in cycle and the current period, identical characteristic point preferably was mapped to overlapping.In this way, mapping device 28 fingerprint image that two-dimensional arrangements is obtained at every turn in virtual image space.At this, virtual image space 29 has the size of the fingerprint image that needs in User Recognition.That is to say preferably have the size of fingerprint image required in User Recognition as the virtual image space 29 that is used for synthesizing at the memory device of the required fingerprint image of User Recognition.For example, virtual image space 29 has the size less than about 100 * 100 pixels.

Whether the size that recognition device 30 is surveyed the whole fingerprint image of mapping in virtual image space 29 equals the size in virtual image space 29, if measure-alike then, then the fingerprint image with whole mapping compares with previous fingerprint image of registering, to confirm the user.

Arithmetic unit 31 receives from the displacement data of mobile sniffer 27 and uses displacement data to calculate pointer with the direction, distance and the mobile degree that move.Arithmetic unit 30 generally combines with pointing device or processor with device of pointing device.As a result, processor can move with required direction and required distance on the screen of display device by steering needle.

In one embodiment, fingerprint deriving means 24 can be presented as variety of way.That is to say that fingerprint deriving means 24 can enough semiconductor device or with aforesaid optical system embodiment.At this, fingerprint deriving means 24 long time that use this optical system just commerce turn to verification system and on scraping, temperature and durability, have superiority.Yet this optical system is owing to the size of optical pickocff has the limitation that only is used in the small portable terminal equipment and has the problem that can not carry out Information Security and identification employing.Simultaneously, use the fingerprint deriving means 24 of semiconductor device when obtaining fingerprint image, to have picture reflection and reaction speed clearly fast.In addition, because the miniaturization of transducer is feasible, use the fingerprint deriving means 24 of semiconductor device to have various ranges of application and on cost, have high competitiveness.

In one embodiment, fingerprint image obtain can enough this optical systems or this semiconductor system carry out.For example, when fingerprint image obtains with semiconductor system, do not need above-mentioned light-emitting device 22 and beam condensing unit 23.As a result, Fig. 3 shows the example that is used for obtaining with optical system the pointing device of fingerprint image, and fingerprint image can obtain by enough this semiconductor systems.The processing method of the fingerprint image that is to obtain because feature of the present invention does not lie in obtaining of fingerprint image is so the fingerprint acquisition methods can enough any systems be carried out.

Hereinafter, the operation to pointing device with fingerprint identification function is described in detail.In one embodiment of the invention, because the execution simultaneously in pointing device of fingerprint identification function and fixed point function, so fingerprint identification function and fixed point function are described.

Fig. 4 shows the view according to the processing of displacement calculating data of the present invention.Fig. 4 a showed the fingerprint image that obtains and characteristic point and Fig. 4 b thereof and show fingerprint image and the characteristic point of obtaining thereof in second round in the period 1.The fingerprint image of Fig. 4 a and 4b is formed in the image in the fingerprint deriving means 24.In one embodiment, as example, show the fingerprint image that extracts 5 characteristic points (being expressed as M).

The fingerprint image of Fig. 4 b is by moving right the fingerprint image of Fig. 4 a 3 pixels (Δ X=3) and move down 3 pixels (Δ Y=-3) and obtain in predetermined period.Mobile sniffer 27 obtains moving of fingerprint image by the displacement data (direction and distance) that calculates the characteristic point of extracting.

Mapping device 28 uses to move to survey adorns the 27 characteristic point displacement datas that calculate, and the fingerprint image that at every turn obtains is shone upon in the correspondence position in virtual image space 29.Be described in detail with reference to 5 pairs of mapping processing of figure.

In one embodiment, use being used for obtaining microminiaturized fingerprint deriving means 24 less than the fingerprint image of 20 * 20 pixels obtains the processing that has at the fingerprint image of the required size of User Recognition (promptly less than about 100 * 100 pixels) and is described.

Fig. 5 a shows use and shows the mapped virtual image space 29 less than 100 * 100 pixels of fingerprint image less than fingerprint image and Fig. 5 b that the microminiaturized fingerprint deriving means 24 of 20 * 20 pixels obtained according to the previous cycle that is provided with.In Fig. 5,, suppose that predetermined pattern is not the shape of fingerprint but the figure of pattern for the convenience that illustrates.

When fingerprint deriving means 24 in time T ₀When obtaining first fingerprint image 41 of 20 * 20 pixels, feature point extraction device 25 extracts one or more at least characteristic points and characteristic point is stored in the storage device 26 from first fingerprint image 41 that obtains.In Fig. 5 a, on first fingerprint image 41, extract 6 characteristic points (being expressed as stain).Mapping device 28 shines upon first fingerprint image 41 on the precalculated position in virtual image space 29.At this, mapping device 28 fingerprint image that preferably mapping is obtained on the center in virtual image space 29.Thereafter, when fingerprint deriving means 24 in time T ₁When obtaining second fingerprint image 42, characteristic point point extraction element 25 extracts one or more at least characteristic points (number: 8) and with characteristic point be stored in the storage device 26 from second fingerprint image 42.

Mobile sniffer 27 uses the mobile message of first fingerprint image 41 and second fingerprint image 42 to come displacement calculating data (direction and distance).Displacement data calculates by the method described in Fig. 4.Mapping device 28 is mapping second fingerprint image 42 in corresponding to the position in the virtual image space 29 of the displacement data that is calculated in the mobile sniffer 27.Then, when fingerprint deriving means 24 in time T ₂When obtaining the 3rd fingerprint image 43, feature point extraction device 25 extracts one or more at least characteristic point (numbers: 9) from the 3rd fingerprint image 43.The characteristic point of extracting is stored in the storage device 26, and uses the characteristic point of extracting to calculate about the displacement data of second fingerprint image 42 and the 3rd fingerprint image 43.Mapping device 28 is mapping the 3rd fingerprint image 43 in corresponding to the position in the virtual image space 29 of the displacement data that is calculated.As mentioned above, fingerprint image obtains, feature point extraction, displacement data are calculated and map operation was repeatedly carried out according to the predetermined cycle, up to the mapping fingerprint image 41,42,43 ..., n overall dimensions be size required in User Recognition, that is to say, the size in virtual image space 29.In this way, having large-sized fingerprint image required in User Recognition can have undersized a plurality of fingerprint image by each and obtain.

During map operation, when the fingerprint image that obtains in current period is shone upon in virtual image space 29, preferred mappings characteristics point is so that at least a portion of the characteristic point of the fingerprint image that obtains in the cycle formerly can be overlapping with the characteristic point of the fingerprint image that obtains in the current period.For example, when shining upon in the virtual image space 29 of second fingerprint image 42 in Fig. 5 b, at least a portion characteristic point of second fingerprint image 42 is mapped to at least a portion of the characteristic point of first fingerprint image 41 overlapping.Similarly, when the 3rd fingerprint image 43 shone upon in virtual image space 29, at least a portion of the characteristic point of the 3rd fingerprint image 43 was mapped to at least a portion of the characteristic point of second fingerprint image 42 overlapping.In Fig. 5 b,, and represent second fingerprint images 42 and the overlapping part of the 3rd fingerprint image 43 with reference to number 49 with reference to number 48 expression first fingerprint images 41 part overlapping with second fingerprint image 42.

Simultaneously, second fingerprint image 42 moves the scheduled time (T1-T0) and obtains by will point 20 after obtaining first fingerprint image 41.In Fig. 3, when finger 20 contact during Transparent Parts 21 with either direction move arbitrary apart from the time, obtain fingerprint image according to the cycle that is provided with.In this case, the moving direction of finger 20 is opposite with the moving direction of fingerprint image.

As mentioned above, the fingerprint image that at every turn obtains in the cycle that is provided with is by mapping in virtual image space 29.When the overall dimensions of the fingerprint image that shines upon equaled the size in virtual image space 29, the whole fingerprint image that recognition device 30 will shine upon in virtual image space 29 compared with the fingerprint image of previous registration, determined then to discern.At this, this identification is preferably determined by the Feature Points Matching of fingerprint image.Recognition device 30 confirms users if two fingerprint images are identical, if but inequality then refusal confirms the user.As a result, the use that the user can limiting terminal equipment is so that have only this user can use or prevent the user to be intended to the information of being leaked in the equipment.

The pointing device of Fig. 6 comprises a plurality of light-emitting devices and the fingerprint deriving means of Duoing than Fig. 3.In Fig. 6, a plurality of fingerprint deriving means 24-1,2,3 are obtaining a plurality of fingerprint images according to predetermined period at every turn.Compare with Fig. 3, when obtaining a fingerprint image in each cycle in the pointing device of fingerprint deriving means 24 at Fig. 3, a plurality of fingerprint deriving means 24-1,2, in 3 each is obtained fingerprint image, so that obtain a plurality of fingerprint images in each cycle in the pointing device of Fig. 6.Fig. 6 shows and comprises 3 light-emitting device 23-1,2,3 and 3 fingerprint deriving means 24-1,2,3 pointing device.

Fig. 7 shows mapping from a plurality of fingerprint deriving means 24-1 shown in Figure 6, the view of the processing of 2,3 fingerprint images that obtain.

Fig. 7 shows the fingerprint image that uses 3 microminiaturized fingerprint deriving means to obtain about 20 * 20 pixels and the fingerprint image that uses about 20 * 20 pixels and obtains have size required in the User Recognition treatment of picture of (about 100 * 100 pixels).Fig. 7 a shows fingerprint deriving means 24-1, the fingerprint image of 20 * 20 pixels that the previous cycle that is provided with of 2,3 bases obtains.Fig. 7 b shows the processing corresponding to the fingerprint image shown in the mapping graph 7a of the position in the virtual image space 29 of 100 * 100 pixels.

When the mapping processing of Fig. 7 is compared with the mapping processing of Fig. 5, the mapping of Fig. 5 is handled a fingerprint image that will at every turn obtain according to the cycle that is provided with and is shone upon in

virtual image space

29, and 3 fingerprint images that the mapping of Fig. 7 processing will be obtained according to the predetermined cycle at every turn shine upon in virtual image space 29.

With reference to figure 7a and 7b, fingerprint deriving means 24-1,2,3 in time T ₀Obtain the fingerprint image (the first fingerprint image image set 61) of 3 20 * 20 pixels simultaneously.Next, feature point extraction device 25 extracts one or more at least characteristic points and the characteristic point of extracting is stored in the storage device 26 from the first fingerprint image image set 61.The characteristic point of the first fingerprint image image set 61 among Fig. 7 a is total up to 12 (being expressed as stain).Mapping device 28 shines upon the first fingerprint image image set 61 in the ad-hoc location in virtual image space 29.At this, the first fingerprint image image set 61 is preferably in the mapping of the center in virtual image space 29.Thereafter, fingerprint deriving means 24-1,2,3 obtain 3 fingerprint images (the second fingerprint image image set 62) in next time T 1.Feature point extraction device 25 extracts one or more at least characteristic points (number: 9) and with characteristic point be stored in the storage device 26 from the second fingerprint image image set 62.Mobile sniffer 27 uses the mobile message of the first fingerprint image image set 61 and the second fingerprint image image set 62 to come displacement calculating data (direction and distance).Displacement data calculates by the same procedure described in Fig. 4.Mapping device 28 shines upon the second fingerprint image image set 62 in the position of the displacement data that is calculated corresponding to mobile sniffer 27.Map operation is repeatedly carried out, up to the mapping fingerprint image image set 61,62 ..., n overall dimensions become the required size of User Recognition, that is to say, the size in virtual image space 29.As a result, having large-sized fingerprint image required in User Recognition can have undersized a plurality of fingerprint image by each and obtain.At this, the second fingerprint image image set 62 will be pointed mobile scheduled time T after being included in and obtaining the first fingerprint image image set 61 ₁-T0 is from 3 fingerprint deriving means 24-1, the fingerprint image that obtains in 2,3.

At this, describe as Fig. 5, when the fingerprint image image set that obtains in current period is shone upon in virtual image space 29, preferred mappings characteristics point is so that the part of the characteristic point of the fingerprint image image set that obtains in current period can be overlapping with the part of the characteristic point of the fingerprint image that obtains in the previous cycle.

When the fingerprint image image set shone upon in whole virtual image space 29, recognition device 30 compared to determine identification by the whole fingerprint image that will shine upon with previous fingerprint image of registering in virtual image space 29.When two fingerprint images were identical, recognition device 30 confirmed the user, but when inequality, recognition device 30 refusals confirm the user.

Simultaneously, pointing device moves steering needle by the fingerprint image that obtains from finger surface according to an embodiment of the invention.Fixed-point processing in the pointing device is as described below.Light-emitting device 22, beam condensing unit 23, fingerprint deriving means 24, feature point extraction device 25, storage device 26 and the operation of mobile sniffer 27 are above-described identical.Yet when carrying out the fixed point function, arithmetic unit 31 receives the displacement data about the characteristic point of fingerprint image that calculates or fingerprint image image set in mobile sniffer 27, and uses displacement data to calculate direction and distance that pointer will move on monitor.That is to say that as shown in Figure 4, arithmetic unit 31 calculates required direction and the required separation distance that pointer will move.At this, though mobile sniffer 27 uses the characteristic point of the fingerprint image that obtains to come the displacement calculating data, displacement data can use the digital finger-print view data directly to calculate.

If n is set to 1 (S802) after the pointing device initialization, each fingerprint deriving means 24 then, 24-1,2,3 obtain the fingerprint image (S803) of n (n=1) 20 * 20 pixels.At this, as a plurality of fingerprint deriving means 24-1,2,3 when being used as shown in Figure 6, and pointing device can obtain a plurality of fingerprint images (fingerprint image image set) simultaneously.As a result, though the fingerprint image that obtains from each fingerprint deriving means is of a size of about 20 * 20 pixels at every turn,, the overall dimensions of the fingerprint image that at every turn obtains can be regulated by the number of control fingerprint deriving means.

Feature point extraction device 25 is from by fingerprint deriving means 24, and 24-1 extracts one or more at least characteristic points and characteristic point is stored in (S804) in the storage device 26 in 2,3 n that the obtains fingerprint image.

Next, mapping device 28 uses the characteristic point of extracting to shine upon n fingerprint image (S805) in virtual image space 29.

Whether the size that recognition device 30 is identified in the virtual image space 29 the whole fingerprint image of mapping becomes the size (S806) of previous setting.At this, the size Expressing of setting required minimum dimension in User Recognition.That is to say, though from each fingerprint deriving means 24,24-1,2,3 each fingerprint image that obtains have the size of about 20 * 20 pixels, the information that is not enough to obtain to be used for User Recognition but the size of fingerprint image is enough to obtain the mobile message of fingerprint image.That is to say that the fingerprint image of 20 * 20 pixels is enough to obtain the mobile message of fingerprint image, but carry out the fingerprint image that User Recognition needs about 100 * 100 pixels by fingerprint image.As a result, the setting of fingerprint image is of a size of about 100 * 100 pixels, and it is the size in virtual image space 29.

If in step S806 in virtual image space 29 size of the whole fingerprint image of mapping less than the previous size that is provided with, that is, and the size in virtual image space 29, then variable n increases progressively 1 (S807), continues to obtain fingerprint image (S803～S806) then.Fingerprint image obtains to handle and proceeds, and reaches the size of previous setting up to the size of the whole fingerprint image of mapping in virtual image space 29.

When the size of the whole fingerprint image that shines upon in virtual image space 29 at step S806 reached the size of previous setting, recognition device 30 extracted one or more at least characteristic points (S808) from the whole fingerprint image of mapping virtual image space 29.

The characteristic point of the whole fingerprint image that recognition device 30 will extract in step S808 compares (S809) with the characteristic point of the fingerprint image of previous registration.

Recognition device 30 is determined in step S808 characteristic point relatively whether identical (S810), and if characteristic point is identical then recognition device 30 confirms users (S811), if but inequality then refusal confirms user (S812).

Figure 9 shows the detail flowchart of processing of mapping fingerprint image in virtual image space (S805) among Fig. 8.

At first, first fingerprint image shines upon (S901) in the ad-hoc location in virtual image space 29.At this, first fingerprint image (or fingerprint image image set) is preferably in the mapping of the center in virtual image space 29.

Then, when second fingerprint image in following one-period by fingerprint deriving means 24,24-1,2,3 when obtaining, and mobile sniffer 27 receives second fingerprint images (S902), with the displacement data (distance and direction) that calculates second fingerprint image and first fingerprint image (S903).At this, second fingerprint image is according to the interior at interval at the fixed time fingerprint image that obtains of moving of fingerprint.The displacement data of step S903 uses the mobile message of the characteristic point of first fingerprint image and second fingerprint image to calculate.Thereafter, mapping device 28 shines upon (S904) according to the displacement data that mobile sniffer 27 is calculated with second fingerprint image in the correspondence position in virtual image space 29.

Fingerprint obtains, displacement data calculates and map operation is carried out n time continuously, reaches the size of previous setting up to the size of whole fingerprint image, that is, the size in virtual image space 29 (S905～S908).

Shown in Fig. 8 and 9, the fingerprint image of about 20 * 20 pixels that obtain for n time according to the cycle that is provided with is synthesized a big fingerprint image, to have size required in User Recognition, for example about 100 * 100 pixels.Image with size required in User Recognition can obtain by synthesize fingerprint image and the relative motion information thereof obtained on each position.

If point 20 contact Transparent Parts 21 (S1001), then the n fingerprint image passes through fingerprint deriving means 24,24-1, and 2,3 obtain (S1002).Then, (n+1) fingerprint image passes through fingerprint deriving means 24,24-1, and 2,3 according to the previous cycles acquisitions (S1003) that are provided with.Mobile sniffer 27 calculates the degree that moves from n fingerprint image to the (n+1) fingerprint image, that is, and and displacement data (S1004).Arithmetic unit 31 uses the coordinate figure of displacement data computing pointer, that is, and and direction that moves and distance (S1005).Next, the coordinate figure of the pointer that is calculated corresponding to arithmetic unit 31, processor (illustrating) combine with arithmetic unit 31 and move this pointer (S1006).

In this way, in Fig. 3 and 6, pointing device shines upon each and has the size that is applicable to pointer control and by fingerprint deriving means 24,24-1,2,3 a plurality of fingerprint images that obtain and it is extended to the size with the User Recognition of being applicable to.As a result, the control of User Recognition and pointer can use a kind of fingerprint Identification sensor to carry out simultaneously.

The pointing device of Figure 11 comprises light-emitting device 22, beam condensing unit 23, fingerprint deriving means 34, fingerprint identification unit 100 and fixed point control unit 200.

Fingerprint identification unit 100 comprises feature point extraction device 35 and recognition device 36, and fixed point control unit 200 comprises fingerprint image extraction element 37, mobile sniffer 38 and arithmetic unit 39.At this, pointing device can further comprise the shell (not shown) according to an embodiment of the invention, shell comprises light-emitting device 22, beam condensing unit 23, fingerprint device 34, feature point extraction device 35, mobile sniffer 38 and arithmetic unit 39, and comprise Transparent Parts 21, wherein finger surface contacts fingerprint deriving means 34 within a predetermined distance.More preferably, the pointing device of Figure 11 suitably is installed in the portable terminal.

In the pointing device of Figure 11, when finger 20 contact Transparent Parts 21, light-emitting device 22 emits light into the surface of finger 20.Light-emitting device 22 comprises at least one or a plurality of light-emitting diode (being abbreviated as " LED ").

Beam condensing unit 23 accumulate in light after light-emitting device 22 directives finger 20 from pointing the light of 20 surface reflection.Common light convex lens can be used as beam condensing unit 23.

Fingerprint deriving means 34 obtains the fingerprint image of finger surface, is used to use the light of assembling by beam condensing unit 23 to come steering needle.The simulation fingerprint image that fingerprint deriving means 34 is assembled beam condensing unit 23 converts the digital finger-print image to, to obtain the fingerprint image of M * N pixel.At this, required size in the size Expressing User Recognition of the M that fingerprint deriving means 34 obtains * N pixel.That is to say that the size Expressing of M * N pixel is carried out the size of User Recognition by using the fingerprint image that once obtains on fingerprint image.Fingerprint obtains to be adorned 34 and comprises optic sensor array, two-dimensional arrangements a plurality of cmos image sensors (being abbreviated as " CIS ").At this, obtain fingerprint image in the cycle that fingerprint deriving means 34 formerly is provided with.Fingerprint deriving means 34 manufactures and is suitable for small portable apparatus, and is used to obtain the CIS of the big fingerprint image that surpasses about 100 * 100 pixels.In this way, as the fingerprint deriving means 34 that is used for User Recognition, CIS can be used in and obtains the fingerprint image with various sizes, generally from the scope of 90 * 90 pixels to 400 * 400 pixels.Therefore, the size of the fingerprint image that obtains of the size of the fingerprint image that obtains of the fingerprint deriving means 34 of the third embodiment of the present invention and the fingerprint deriving means 24 of of the present invention first or second embodiment is different.

The light that produces from light-emitting device 22 is launched into the surface of finger 20 and reflects according to the figure on the surface of finger 20.Forming one by beam condensing unit 23 from the light of the basal surface reflection of finger fingerprint deriving means 34 resembles.Be formed on resembling in the fingerprint deriving means 34 and be converted to the digital finger-print image by fingerprint deriving means 34.This fingerprint image obtains on time shaft and carries out continuously apace.

Fingerprint identification unit 100 is at operation cycle extract minutiae from the fingerprint image that fingerprint deriving means 34 obtains that is different from fingerprint deriving means 34, and the characteristic point by the characteristic point that will extract and previous registered fingerprint image compares and carries out User Recognition.Fingerprint identification unit 100 with the characteristic point of the characteristic point extracted and the fingerprint image of previous registration compare into general per second once to three times.That is to say, receive 1-3 fingerprint image, compare from the fingerprint image extract minutiae that receives with the characteristic point of the characteristic point extracted and the fingerprint image of previous registration and carry out the fingerprint recognition processing that is used to confirm the user by per second.More preferably, per second is carried out the processing of fingerprint recognition.The characteristic point that fingerprint identification unit 100 comprises the fingerprint image that is used for from the feature point extraction device 35 of the fingerprint image extract minutiae that obtains and is used for feature point extraction device 35 is extracted compares the recognition device 36 of carrying out User Recognition with the characteristic point of the fingerprint image of previous registration.

Fixed point control unit 200 extracts m * n pixel from the fingerprint image that fingerprint deriving means 34 is obtained in the operation cycle that is different from fingerprint identification unit 100 (fingerprint image n) is to survey the mobile message of fingerprint image for M, N＞m.Fixed point control unit 200 uses the mobile message of surveying to come the displacement calculating data and uses the displacement data that calculates to calculate pointer with the direction and the distance that move.Preferably, fixed point control unit 200 survey fingerprint images characteristic point mobile message and according to the displacement data of mobile message calculated characteristics point.Corresponding to the displacement data of characteristic point, fixed point control unit 200 calculates the moving direction and the distance of pointer.

Extract the fingerprint image of about 20 * 20 pixels in the fingerprint image that fixed point control unit 200 obtains from the cycle that formerly is provided with, calculate the displacement data of each fingerprint image and use displacement data to calculate two-dimensional coordinate (Δ X, Δ Y), that is, pointer is with the two-dimensional directional and the distance that move.Fixed point control unit 200 about per seconds carry out 800～1200 times fingerprint image extraction and calculate the displacement data of each fingerprint image that extracts according to the cycle of correspondence.

Fingerprint identification unit 100 is operated respectively according to the different operation cycles with fixed point control unit 200, to carry out User Recognition and pointer control operation.That is to say that when the user of pointing device used the fingerprint image steering needle, fingerprint identification unit 100 was independent of the fingerprint authentication of fixed point control and treatment execution to the user.As a result, during the navigation to pointer control, fingerprint authentication is carried out periodically, and does not have other fingerprint recognition to handle.

Hereinafter, the operation to fingerprint identification unit 100 and fixed point control unit 200 is described in detail.

Feature point extraction device 35 extracts one or more at least characteristic points from the previous each fingerprint image that obtains of cycle that is provided with of basis.These characteristic points comprise the lines length and the direction of fingerprint image and lines separates or the position data of termination.

The characteristic point of the fingerprint image that recognition device 36 will extract from feature point extraction device 35 compares with the characteristic point of the fingerprint image of previous registration, to carry out User Recognition according to the evaluation of two fingerprint images.At this, recognition device 36 can comprise the comparison means (not shown), is used to make up fingerprint image that obtains and the fingerprint image global information of before having registered and local characteristic information or the Feature Points Matching of two fingerprint images is come these two fingerprint images of comparison.Recognition device 36 use comparison means are determined the evaluation to two fingerprint images.

If the characteristic point of the fingerprint image of previous registration was identical with the characteristic point of the fingerprint image that extracts from feature point extraction device 35, recognition device 36 execution User Recognition then are not if perhaps have identical then refuse this User Recognition.

Extract the fingerprint image of the M that fingerprint image extraction element 37 obtains from fingerprint deriving means 34 * N pixel m * n pixel (at this M, N＞m, fingerprint image n).The size that the size Expressing of m * n pixel is used in pointer control.The size of 20 * 20 pixels that is used for the pointer control of pointing device is enough to realize generally extracting the fingerprint image of little fingerprint image.The fingerprint image that fingerprint image extraction element 37 extracts from about 15 * 15 pixels to about 80 * 80 pixels.As a result, the fingerprint image of about 100 * 100 pixels of obtaining from the fingerprint deriving means 34 that is used for User Recognition of fingerprint image extraction element 37 fingerprint image that extracts about 20 * 20 pixels is used for pointer control.At this, the size of employed fingerprint image is an example of the present invention.That is to say, be applicable to User Recognition the gravel size decision of the M that is obtained * N pixel, and the gravel size decision of the m that is extracted * n pixel be applicable to pointer control.

Mobile sniffer 38 obtains the mobile degree according to each each fingerprint image that obtains of cycle that is provided with.At this, mobile sniffer 38 is preferably by calculating the displacement data relevant with the characteristic point of the fingerprint image that obtains (direction and distance) and using mobile estimating method to survey the mobile degree of fingerprint in the cycle that is provided with.More preferably, mobile sniffer 38 is by calculating the mobile degree of surveying fingerprint image in the cycle that is provided with about the displacement data of the characteristic point of the fingerprint image that obtains.At this, the displacement data of the fingerprint image preferably characteristic point by calculating the fingerprint image obtain from the cycle formerly calculates to the displacement and the direction of the characteristic point of the fingerprint image that obtains in current period.It is important factor that the mobile message of the fingerprint image in the fingerprint recognition and feature point extraction and fingerprint obtain, because how the reliability mobile and fingerprint recognition of fingerprint image can be extracted to distinguish according to characteristic point reliably comes.

Arithmetic unit 39 receives the mobile degree of fingerprint images from mobile sniffer 38, the shift certificate of ascending the throne, and use the displacement data that receives to calculate two-dimensional coordinate (Δ X, Δ Y), promptly pointer is with the direction and the distance/mobile degree that move.

Arithmetic unit 39 generally combines with pointing device or processor with device of pointing device.As a result, processor can be controlled at moving of pointer on the screen of display apparatus according to the coordinate that calculates in arithmetic unit 39.

Pointing device according to an embodiment of the invention can further comprise the display unit (not shown) that is used to show previously stored information.In this way, when pointing device further comprised display unit, this display unit was come received signal according to the fingerprint recognition of fingerprint identification unit 100, to show recognition result.When being identified in of user successfully carried out in the fingerprint identification unit 100, the information that is used to carry out all functions of counterpart terminal was presented at this display unit.Yet, when User Recognition is rejected, showing restrictive information, it only can carry out the specific function of this terminal.

Restrictively allow the technology of the use of counterpart terminal mentioning after a while by User Recognition.

Fingerprint deriving means 34 in the third embodiment of the present invention can embody by enough semiconductor device shown in first and second embodiment of the present invention.

Simultaneously, when fingerprint deriving means 34 is presented as optical system, the enough small fingerprint deriving means of energy, promptly " contracting and resemble optical system " obtains to be used for the big fingerprint image of User Recognition.In other words, the size of fingerprint deriving means 34 can reduce 1/2～1/4 and obtain the fingerprint image that reduces and come miniaturization by the size with actual fingerprint.

Detailed description is used to contract and is resembled principle and the processing that optical system (reduced optical system) is obtained fingerprint image now.

Figure 12 shows 3: 1 of being installed in the trickle space according to the present invention and contracts and resemble the view of Design for optical system example.As shown in figure 12, aspherical mirror 42 is used for optics is installed in the short space.If optical system configurations is for as shown in figure 12, the size of the actual object of then left arrow 41 representatives is reduced to about 1/3 size of right arrow 43 representatives.If the image of the object of left arrow 41 representatives through aspherical mirror 42, then forms the inverted image of about 1/3 size at the fingerprint deriving means 34 that is arranged in the right side.Contracting resembles optical system and embodies by using above-mentioned principle, so that the size of actual fingerprint is reduced to the size (being from 1 to 5 real number at this n) of 1/n.

Figure 13 shows by using contracting of Figure 12 and resembles the view that optical system is come the example of the fingerprint image that obtains from the fingerprint deriving means.It is that the fingerprint image that from fingerprint deriving means 34 obtains at 1: 1 and Figure 13 b show contracting that to resemble optical system be the fingerprint image that from fingerprint deriving means 34 obtains at 4: 1 that Figure 13 a shows in optical system.

Generally speaking, every 1mm forms about two ditches (valley) in people's fingerprint.As a result, when using the fingerprint deriving means 34 of 20 * 20 pixels, the identification pixel of fingerprint deriving means 34 is 0.5mm, and the fingerprint number that fingerprint deriving means 34 obtains is for just 2, shown in Figure 13 a.In this way, when the number of the ditch of the fingerprint image that is obtained diminished, the accuracy of User Recognition descended, and the performance of pointer control also descends.As a result, the size of transducer needs bigger, to carry out enough data collections.

In order to overcome the problems referred to above, can resemble optical system and obtain more data by fingerprint deriving means 34 being used contract, and need not to increase the size of transducer.That is to say, in an embodiment of the present invention, obtain and have the size of the fingerprint of 0.5mm at interval by the size of fingerprint being dwindled 1/n (be at this n 1 to 5 real number).More particularly, the size of fingerprint is reduced to 1/2～1/4 size.As a result, can use the fingerprint deriving means 34 that has with Figure 13 a same size to obtain the data of Duoing than Figure 13 a.Shown in Figure 13 b, contracting when use resembles optical system when making the size of fingerprint be reduced to 1/4 size, and the fingerprint of 0.5mm can be reduced to about 0.125mm at interval.Therefore, can use the fingerprint deriving means 34 that has with Figure 13 a same size to obtain 4～16 times finger print information.In other words, when being reduced to 1/2～1/4 size by the fingerprint that will have equispaced 0.5mm when obtaining fingerprint image, the size of fingerprint deriving means 34 can be miniaturized to 1/4～1/16.As a result, can use low energy consumption and cheaply miniaturization fingerprint deriving means 34 obtain to be used for the fingerprint image of User Recognition and pointer control, and miniaturization fingerprint deriving means 34 is favourable when being applied to the small portable terminal equipment.

Figure 14 shows the view of the pointing device with fingerprint identification function of a fourth embodiment in accordance with the invention.

Compare with Figure 11, the pointing device of Figure 14 further comprises storage device 60.The fingerprint image that storage device 60 storages are obtained from fingerprint deriving means 34.In the pointing device of Figure 14, if the fingerprint image that obtains from fingerprint deriving means 34 according to the cycle of previous setting is stored in the storage device 60 at every turn, then fingerprint identification unit 100 and fixed point control unit 200 use the fingerprint image that is stored in the storage device 60 to carry out User Recognition and pointer control respectively.That is to say, when in fingerprint identification unit 100 and fixed point control unit 200, carrying out User Recognition and pointer control respectively, it receives the fingerprint image that obtains according to the operation cycle of fingerprint deriving means 34 immediately in the pointing device of Figure 11, the fingerprint image that obtains from fingerprint deriving means 34 at first is stored in the storage device 60, only use fingerprint image to carry out pointer control then, generate so that pointer also can be presented as low cost, low energy consumption and high speed navigation information with the required size of pointer control.At this, use the fingerprint image that is stored in the storage device 60 to carry out User Recognition and pointer control simultaneously naturally.

In this way, the fingerprint image that obtains by the fingerprint deriving means 34 that is used for User Recognition from the pointing device of Figure 11 and 14 extracts the fingerprint image with size required in the pointer control, and User Recognition and pointer control can use a kind of fingerprint Identification sensor to carry out simultaneously.

Fingerprint deriving means 34 obtains the fingerprint image 71 of M * N pixel according to predetermined period.The fingerprint image 71 of M * N pixel has the size that is enough to be used in User Recognition.Preferably, fingerprint image 71 has from the size of 90 * 90 pixels to 400 * 400 pixels.In addition, fingerprint image extraction element 37 extracts the fingerprint image 72 of m * n pixel from the fingerprint image 71 of M * N pixel.At this, fingerprint image extraction element 37 extracts the core of the fingerprint image 71 of M * N pixel.The size Expressing of m * n pixel can be carried out the size of the fingerprint image 72 of pointer control.Fingerprint image 72 has from the size of 15 * 15 pixels to 80 * 80 pixels.

In the pointing device of Figure 14, the fingerprint image 71 of M * N pixel is stored in the storage device 60, and User Recognition uses the fingerprint image 71 of M * N pixel to carry out.Simultaneously, pointer control uses the fingerprint image 72 of the m * n pixel that extracts from the fingerprint image 71 of M * N pixel to carry out.

In the situation that the fingerprint image 72 of pointer use m * n pixel is regulated, when Δ X and Δ Y are moved in finger 20 surface from the primary importance to the second place, be sent to mobile sniffer 38 about the data of the fingerprint image of m * n pixel of extracting from fingerprint image extraction element 37.At this, fingerprint image is sent with about 800～1200 times speed of per second.As a result, depend on that the displacement data of the fingerprint image that moves of finger 20 is calculated and be converted to speed, and the moving direction of pointer and distance also calculate and are converted to speed.At this,,, handle and amount of calculation to reduce so preferably select minimum picture size because above-mentioned processing speed need be used for stable fixed-point operation in pointing device according to an embodiment of the invention.

Simultaneously, the whole fingerprint image 71 of the required M * N pixel of User Recognition is sent to fingerprint identification unit 100.Fingerprint image 71 sends with the speed of per second 1～3 time, wherein can carry out general identification and handle.Fingerprint identification unit 100 is configured to be included in the treatment facility of portable terminal, so that fingerprint identification unit 100 can be carried out this function.

Use the characteristic point of fingerprint image 72 to come the processing of displacement data of calculated fingerprint image 72 identical with Fig. 4.

At init state (S1601),, then obtain the fingerprint image (S1603) of M * N pixel by fingerprint deriving means 34 according to first operation cycle if point 20 surface contact Transparent Parts 21 (S1602).

The fingerprint image 71 that fingerprint identification unit 100 and fixed point control unit 200 use fingerprint deriving means 34 to obtain is carried out User Recognition (S1620) and pointer control (S1630) simultaneously.In the situation of the pointing device of Figure 14, except the fingerprint image 71 that obtains is stored in the storage device 60 and uses fingerprint image 61 fingerprints that are stored in the storage device 60, it is identical with the pointing device of Figure 11 that User Recognition (S1620) and pointer are controlled (S1630).

In User Recognition place (S1620), feature point extraction device 35 extracts one or more at least characteristic points according to second operation cycle from the fingerprint image of M * N pixel, to send characteristic point to recognition device 36 (S1604).The characteristic point that recognition device 36 will extract from the fingerprint image of M * N pixel compares (S1605) with the characteristic point of the fingerprint image of previous registration.Whether the characteristic point that recognition device 36 is determined two fingerprint images from comparative result identical (S1606).If the characteristic point of two fingerprint images is identical, then recognition device 36 confirms users (S1607), if difference then refusing user's identification (S1608).

Next, in the pointer control and treatment in (S1630), fingerprint image extraction element 3 extracts the fingerprint image 72 of m * n pixel according to the period 3 from the fingerprint image of M * N pixel, to send the fingerprint image that extracts to mobile sniffer 38 (S1609).In the present embodiment, m and n are from 15 to 80 scope, with the size of the pointer control of fingerprint image 72 with the extraction of being applicable to.Mobile sniffer 38 calculates the displacement data of the fingerprint image 72 of m * n pixel, to send displacement data to arithmetic unit 39 (S1610).At this, mobile sniffer 38 comes the displacement calculating data by the mobile degree (being distance and direction) of the fingerprint image that calculates the fingerprint image obtain obtain in the current period from the cycle formerly.Preferably, calculate the displacement data of mobile degree of the characteristic point of the fingerprint image 72 depend on extraction.Arithmetic unit 39 uses the displacement data that calculates in mobile extraction element 38 to come the computing pointer with the coordinate (S1611) that moves.Corresponding to the coordinate of pointer, the processor (not shown) moving hand of terminal equipment.

Describe as Figure 16, pointing device can obtain the fingerprint image 71 that transducer 34 obtains from fingerprint and carries out User Recognition and pointer control simultaneously by using according to an embodiment of the invention.The different operating that User Recognition processing (S1620) and pointer control and treatment (S1630) formerly are provided with was carried out in the cycle, and these two treatment S 1620 and S1630 carry out respectively.That is to say that when the user uses fingerprint to regulate pointer (S1630), this User Recognition is handled (S1620) and carried out naturally.As a result, the user be need not to carry out fingerprint recognition handle, and automatically discerned,, can regulate the service range that the user can use corresponding equipment so that according to the fingerprint recognition result at pointer control period fingerprint.

Compare with the above embodiment of the present invention, the 5th embodiment is characterised in that and is not included in the processing of extract minutiae from fingerprint image during the fingerprint identification function.That is to say, do not determine that the fingerprint image that obtains is stored in position in the storage device according to the characteristic point of extracting.Otherwise according to displacement, the shift of ascending the throne is stored in the storage device according to the mapping position of determining fingerprint image and its.

Hereinafter, the structure to the pointing device of Figure 17 is described in detail.

The pointing device of Figure 17 comprises Transparent Parts 21, light-emitting device 22, beam condensing unit 23 and fingerprint deriving means 34.Yet, save explanation, because its identical with the 3rd or the 4th embodiment to the structure of these elements.

The fingerprint image that obtains by fingerprint deriving means 34 is input in the fixed point control unit 200 that comprises fingerprint image extraction element 37, mobile sniffer 38 and arithmetic unit 39 immediately.Because the operation of the concrete element of fixed point control unit 200 is identical with the above-mentioned the 3rd or the 4th embodiment's, so omit specific description.

Yet, compare with the 3rd or the 4th embodiment, the operation of the 5th embodiment is characterised in that the fingerprint image of the m * n pixel that extracts by fingerprint image extraction element 37 is stored in the storage device 40 and the memory location is shone upon according to the data of the shift value that calculates by arithmetic unit 39.Specifically, suppose that the i fingerprint image that extracts is stored in the ad-hoc location of storage device 40 from fingerprint image extraction element 37.If from fingerprint image extraction element 37, extract (i+1) fingerprint image, then receive the displacement data Δ X and the Δ Y that obtain by mobile sniffer 38 and arithmetic unit 39, and (i+1) finger print data is stored in from the ad-hoc location of storing the i finger print data and has moved the position of displacement data.

In storage device 40 operation of storage finger print data by according to predetermined time interval periodically data storing method carry out or when receiving particular command, carry out.

In addition, the fifth embodiment of the present invention comprises CPU 50, be used for controlling the operation that finger print data is stored in above-mentioned storage device 40, control moving of pointing device and carry out fingerprint recognition operation described in Figure 18 by receiving from the displacement data Δ X of arithmetic unit 39 and Δ Y.

If this system is initialised and points 20 surface contact Transparent Parts 21 (S1810), then obtain the fingerprint image (S1820) of m * n pixel by fingerprint deriving means 34 and fingerprint image extraction element 37.

The fingerprint image that obtains is stored in (S1830) in the storage device 40, and the displacement data on the calculated fingerprint image and pointer coordinate (S1840, S1850).Result of calculation is provided for storage device 40, and is used to shine upon fingerprint image.

Result of calculation also offer CPU 50 and be used to control the operation of pointing device (S1860, S1870).

Simultaneously, CPU 50 receptions are from the fingerprint image of storage device 40.If the fingerprint image that receives is not the fingerprint image of M * N pixel, then CPU 50 receives the fingerprint image (S1880) from storage device 40 once more.If the fingerprint image that receives is identical with the fingerprint image of M * N pixel, then fingerprint image is compared (S1890) with previously stored fingerprint image, and determine evaluation (S1899) these two fingerprint images.Then, carry out User Recognition or the operation of identification refusal according to the result.

In one embodiment, has the portable terminal identification user's of fingerprint identification function fingerprint, the user is carried out fingerprint recognition (S1900).Portable terminal is discerned by fingerprint recognition and is intended to use whether the people of this terminal equipment is user oneself (S1910).When the user was user oneself, the whole menu (S1920) that terminal equipment display terminal can provide was so that the user can use all functions (service) (S1930).Yet when the user was not user oneself, terminal equipment only showed the certain menu (S1940) that can allow this people to use.In this way, the individual that can not be identified via fingerprint recognition by restriction such as the important information of credit card and financial information is protected to the use of terminal equipment.

User if not user oneself is intended to use unallowed function (S1950), and then terminal equipment shows the message (S1960) of representing to use corresponding function.Then, after several seconds, terminal equipment shows the menu (S1940) that allows this people to use once more.Therefore, not under user's the situation of previous registration a people, user's personal information, the control information of system and can be protected by the visit that limits other users such as the paid service of ecommerce are with the protection personal information.

Fingerprint recognition is handled can be as background process, the inconvenience that causes with the use that alleviates mobile terminals equipment.At this, when the user used two-dimentional pointer with finger, background process was automatically carried out collection, analysis and is identified the step of desired data in the fingerprint recognition, and not to these steps of user notification.When in user's using system previous when specifying the processing that is used to protect maybe when carrying out the approval transaction, in the data conditions that user oneself use obtains, carry out required processing from background process.Yet, when the user is not user oneself, refuse this processing, with protection owner's information or all.

That is to say, combine with background process and carry out the individual identification processing.When the user is user oneself, carry out next step in succession, but the follow-up use of refusal when the user is not user oneself.As a result, can guarantee required stability, and not influence user's convenience.At this, use fingerprint registration identical and the sensor cluster in the evaluation, and pointing device is configured to carry out simultaneously data collection and software processes with two-dimentional pointing device.

In the above-described embodiments, if user oneself registration his or his fingerprint, then Deng Ji data are stored in the nonvolatile memory and repeatedly use, up to these data of user oneself change.

In another embodiment of the present invention, when the user uses mobile terminals equipment, not to be used for the fingerprint Identification sensor of fingerprint recognition but to have fingerprint confirming whether the pointing device of sensor function can be discerned the portable terminal user by user's fingerprint recognition is user oneself.Operation hereto, when the user moves his or his finger, portable terminal is collected two-dimentional mobile message, produce two dimensional image and extract minutiae from the fingerprint image of correspondence by synthetic collected mobile message and the fingerprint image in the correspondence position with the required size of individual identification.Then, the characteristic point of portable terminal registration extraction maybe compares characteristic point of extracting and the data that are used for the registration of individual identification.

Figure 20 shows the view that comprises the example of pointing device portable terminal according to of the present invention.Portable terminal comprises cell phone, PDA or smart phone.

In portable terminal, expose the outer surface of Transparent Parts 230, and when finger was placed on the outer surface of Transparent Parts 230, the fingerprint image with the required size of User Recognition obtained by fingerprint and obtains.If the acquisition fingerprint image, then existing menu window is converted to service screen 240 as shown in figure 20.

In portable terminal service, the menu that this service can be to use pointer 250 to select and click on the service screen 240 by the mouse moving key that does not use such as all-purpose computer uses.In addition, portable terminal comprises at least one or a plurality of function button 220 that is used to carry out other functions or the order of input performance.

Though the present invention can carry out various modifications and replacement form, specific embodiment is shown in accompanying drawing and the mode of describing in detail with example.Yet, should be appreciated that the present invention is not limited to disclosed concrete form.Otherwise the present invention covers all modifications in the spirit and scope of the present invention defined in the claims, is equal to and replaces.

Commercial Application

Have the pointing device of fingerprint identification function and refer to according to an embodiment of the invention In the line recognition methods, it is mapped to have a undersized fingerprint image, with produce big fingerprint image or The little fingerprint image that extracts from big fingerprint image is so that pointing device is carried out User Recognition and referred to Pin control. As a result, each sensor that is used for User Recognition and is used for pointer control is not included in In the pointing device, but for a kind of sensing of carrying out two functions of User Recognition and pointer control Device is included in the pointing device according to an embodiment of the invention. In addition, owing to need The User Recognition of big fingerprint can only be carried out with little fingerprint image, can easily embody little The portable terminal of type, thus cost reduced.

In addition, can be by limit the service kind that in terminal device, to use according to User Recognition Class is protected the important information in the terminal device.

Claims

1. the pointing device with fingerprint image recognition function comprises:

At least one or a plurality of fingerprint deriving means are used for obtaining according to predetermined period the fingerprint image of finger surface;

The feature point extraction device is used for extracting one or more at least characteristic points from the fingerprint image that obtains;

Mobile sniffer is used to calculate the displacement data between the characteristic point of fingerprint image of extraction, to survey the mobile message of fingerprint image;

Mapping device is used for shining upon fingerprint image according to mobile message in inner virtual image space;

Recognition device, when the overall dimensions of the fingerprint image that shines upon reached the size of previous setting, recognition device compared the fingerprint image of whole mapping with previous fingerprint image of registering, and determined the identification to fingerprint; And

Arithmetic unit, the displacement data and the use displacement data that are used to receive from mobile sniffer calculate pointer with the direction and the distance that move.

2. pointing device as claimed in claim 1, further comprise shell, described shell comprises fingerprint deriving means, feature point extraction device, mobile sniffer, mapping device, recognition device and arithmetic unit, and comprise the Transparent Parts with plane surface that finger surface is in contact with it, described Transparent Parts and fingerprint deriving means have predetermined distance.

3. pointing device as claimed in claim 1 or 2, wherein the fingerprint deriving means is a cmos image sensor.

4. pointing device as claimed in claim 1 or 2, the size in fingerprint image that wherein obtains and virtual image space are respectively m * n pixel and M * N pixel, and m and n are respectively less than M and N.

5. pointing device as claimed in claim 1 or 2, wherein mobile sniffer calculate displacement and the direction of the characteristic point of the fingerprint image that obtains to the characteristic point of the fingerprint image that obtains in current period from the cycle formerly.

6. pointing device as claimed in claim 1 or 2, wherein mapping device shines upon fingerprint image in virtual image space, when between the characteristic point of n-1 fingerprint image and n fingerprint image, having same characteristic features point with box lunch, overlapping identical characteristic point.

7. as claim 1 or 2 described pointing devices, wherein the characteristic point with the fingerprint image of previous registration is identical for the characteristic point of the fingerprint image of recognition device by the characteristic point of the fingerprint image of whole mapping and the characteristic point of the fingerprint image of previous registration being mated determine whole mapping, and according to determining the identification of result's judgement to fingerprint.

8. portable terminal, it has comprised as claim 1 or 2 described pointing devices and has carried out user's the fingerprint recognition and the control of pointer.

9. method that is used to discern the fingerprint that User Recognition uses may further comprise the steps:

The fingerprint image obtaining step is used to use predetermined fingerprint to obtain transducer and obtains at least one or a plurality of fingerprint image according to the cycle that is provided with;

The feature point extraction step is used for extracting one or more at least characteristic points from the fingerprint image that obtains;

First mapping step is used for ad-hoc location mapping first fingerprint image in virtual image space;

The displacement data calculation procedure is used for calculating the displacement data between the characteristic point of second fingerprint image that the characteristic point of first fingerprint image and the following one-period after the cycle of obtaining first fingerprint image obtain;

Second mapping step is used for using displacement data at virtual image space mapping second fingerprint image; And

The fingerprint recognition step, be used for when the overall dimensions of the fingerprint image that shines upon in virtual image space reaches the size of previous setting, the characteristic point of the fingerprint image of the whole mapping characteristic point with the fingerprint image of previous registration is compared, and determine identification fingerprint.

10. method as claimed in claim 9, the size in fingerprint image that wherein obtains and virtual image space are respectively m * n pixel and M * N pixel, and m and n are respectively less than M and N.

11. method as claimed in claim 9, wherein the displacement data calculation procedure is displacement and the direction of calculating from the characteristic point of first fingerprint image to the characteristic point of second fingerprint image.

12. method as claimed in claim 9, wherein second mapping step is mapping second fingerprint image in the corresponding position of the displacement data that is calculated with first fingerprint image that shines upon in virtual image space distance.

13. as claim 9 or 12 described methods, wherein second mapping step is mapping second fingerprint image in virtual image space, when having same characteristic features point between the characteristic point with convenient first fingerprint image and second fingerprint image, and overlapping identical characteristic point.

14. as claim 9 or 12 described methods, wherein the fingerprint image obtaining step is to use a plurality of fingerprints to obtain transducer and obtains a plurality of fingerprint images at every turn.

15. method as claimed in claim 14, wherein a plurality of fingerprint images that obtain are images of adjacent fingerprint.

16. method as claimed in claim 9, wherein the fingerprint recognition step comprises:

First step determines whether the size of the whole fingerprint image that shines upon reaches the size of previous setting in virtual image space;

Second step when reaching the size of previous setting according to the whole fingerprint image of definite result, is extracted one or more at least characteristic points from whole fingerprint image;

Third step compares the characteristic point of extraction and the characteristic point of previous fingerprint image of registering; With

The 4th step is according to the definite identification to fingerprint of comparative result.

17. the pointing device with fingerprint identification function comprises:

Fingerprint deriving means (first operation cycle) is used to obtain the required fingerprint image of finger surface, and described finger surface obtains steering needle by a two dimensional image only;

Fingerprint identification unit (second operation cycle) is used for extracting one or more at least characteristic points from the fingerprint image that obtains, and the characteristic point the obtained characteristic point with the fingerprint image of previous registration is compared, the user of the fingerprint image that obtains with identification; And

Fixed point control unit (the 3rd operation cycle), the partial data that is used for the image that obtains according to first operation cycle are surveyed mobile message and are come the displacement data of calculated fingerprint image according to mobile message, to calculate the moving direction and the distance of pointer.

18. pointing device as claimed in claim 17, wherein fingerprint identification unit comprises:

The feature point extraction device is used for extracting one or more at least characteristic points from the fingerprint image that the fingerprint deriving means obtains; And

Recognition device, the characteristic point that is used for fingerprint image that the feature point extraction device is extracted compares with the characteristic point of the fingerprint image of previous registration, to determine the identification to the user of fingerprint image.

19. pointing device as claimed in claim 18, wherein recognition device mates to determine with the characteristic point of the fingerprint image of before having registered whether the characteristic point of previous fingerprint image of registering is identical with the characteristic point of extraction by the characteristic point that will extract, and according to determining that the result carries out the identification to the user.

20. pointing device as claimed in claim 17, the control unit of wherein fixing a point comprises:

Fingerprint image extraction element, the fingerprint image that is used for M * N pixel (is integer at this M and N) of obtaining from the fingerprint deriving means extract m * n pixel, and (at this, m and n are integers, m＜M, the fingerprint image of n＜N);

Mobile sniffer is used to calculate the displacement data of fingerprint image of the m * n pixel of extraction, to survey the mobile message of each fingerprint image; And

Arithmetic unit is used to receive from the displacement data of mobile sniffer and calculates the moving direction and the distance of pointer according to displacement data.

21 pointing devices as claimed in claim 20, wherein mobile sniffer calculates displacement and the distance of the characteristic point of the fingerprint image that obtains to the feature of the fingerprint image that obtains in current period from the previous cycle, with the displacement data of calculated fingerprint image.

22. pointing device as claimed in claim 20, wherein the scope of M and N from 90 to 400, and the scope of m and n from 15 to 80.

23. pointing device as claimed in claim 17, wherein the fingerprint deriving means is a cmos image sensor.

24. pointing device as claimed in claim 17, wherein the fingerprint deriving means is the active capacitor transducer.

25. pointing device as claimed in claim 17, wherein second operation cycle was 1～3 time/second.

26. pointing device as claimed in claim 17, wherein the 3rd operation cycle was 800～1200 times/second.

27. pointing device as claimed in claim 17, wherein fingerprint identification unit and fixed point control unit are according to second operation cycle and the operation simultaneously respectively of the 3rd operation cycle.

28. pointing device as claimed in claim 17 further comprises:

Light-emitting device is used for the finger surface of light emission to steering needle; With

Beam condensing unit is used to assemble the fingerprint image from the finger surface reflection,

Wherein the fingerprint image assembled of beam condensing unit obtains by the fingerprint deriving means.

29. pointing device as claimed in claim 28, wherein beam condensing unit between light-emitting device and fingerprint deriving means,

The ratio of the distance between distance between light-emitting device and the beam condensing unit and beam condensing unit and the fingerprint deriving means is n: 1, and

N is from 1 to 5 real number.

30. pointing device as claimed in claim 28, wherein beam condensing unit is an aspherical mirror.

31. a portable terminal comprises pointing device as claimed in claim 17 and is used for carrying out simultaneously fingerprint recognition and pointer control.

32. portable terminal as claimed in claim 31 further comprises the display unit that is used to show previously stored information,

Wherein when User Recognition was carried out in fingerprint identification unit, display unit showed whole information or the execution function that the user is allowed, and when not carrying out User Recognition, display unit only shows the interior information and the available function of limited range of previous setting.

33. fix-point method with fingerprint identification function, this fix-point method is the fix-point method of pointer control appliance, the pointer control appliance has size than the also little imageing sensor of predetermined pictures size of images required in fingerprint recognition, said method comprising the steps of:

The fingerprint image obtaining step is used to use predetermined fingerprint to obtain transducer, obtains at least one or a plurality of fingerprint image of M * N pixel on the finger surface of the removable pointer of control according to first operation cycle;

Identification step, be used for according to second operation cycle by comparing to determine identification to the user of fingerprint image from the fingerprint image extract minutiae that obtains and with the characteristic point of the characteristic point extracted and the fingerprint image of previous registration;

The fingerprint image extraction step is used for extracting from the fingerprint image that obtains according to the 3rd operation cycle the fingerprint image of m * n pixel;

Mobile detection steps, the displacement data that is used for the fingerprint image by calculating each m * n pixel extracted is surveyed the mobile message of each fingerprint image; And

Calculation step is used to use displacement data calculating and output pointer with the direction and the distance that move.

34. fix-point method as claimed in claim 33, wherein identification step comprises:

When carrying out User Recognition, the signal of the previous whole information that is provided with is used in output; And

When not carrying out User Recognition, the signal of the previous limited information that is provided with is used in output.

35. fix-point method as claimed in claim 33, wherein the scope of M and N from 90 to 400, and the scope of m and n from 15 to 80.

36. fix-point method as claimed in claim 33, wherein the 3rd operation cycle was 800～1200 times/second.

37. fix-point method as claimed in claim 33, wherein mobile detection steps is calculated displacement and the direction from the k-1 fingerprint image to the characteristic point of k fingerprint image, to calculate the displacement data of each fingerprint image.

38. fix-point method as claimed in claim 33, wherein image extraction step, mobile detection steps and calculation step were carried out from identification step respectively according to the 3rd operation cycle.

39. fix-point method as claimed in claim 33 wherein also comprises step:

Luminous step is used for the finger surface of light emission to steering needle; With

Agglomeration step is used to use aspherical mirror to assemble the fingerprint image that finger surface produces,

Wherein the fingerprint obtaining step is to obtain the fingerprint image that agglomeration step is assembled.

40. fix-point method as claimed in claim 39, wherein agglomeration step is to assemble fingerprint image by via regulating aspherical mirror the size of the fingerprint on the finger surface being reduced 1/n.

41. fix-point method as claimed in claim 40, wherein n is from 1 to 5 real number.

42. the pointing device with fingerprint identification function comprises:

At least one or a plurality of fingerprint deriving means are used for obtaining according to predetermined period or requirement once in a while the image of finger surface;

Arithmetic unit is used to use the fingerprint image that obtains to come the displacement calculating data, calculates pointer with the direction and the distance that move to use displacement data;

Storage device is used for the amount of movement corresponding to the displacement data that receives from arithmetic unit, the fingerprint image that mapping obtains from the fingerprint deriving means; With

CPU is used to analyze and handle the data of arithmetic unit and storage device.

43. pointing device as claimed in claim 42, wherein CPU and arithmetic unit and storage device form an assembly.

44. pointing device as claimed in claim 42, wherein CPU forms the structure of separating with arithmetic unit and storage device.

45. each the described pointing device as in the claim 42 to 44 further comprises:

Light-emitting device is used for the light emission to finger surface; With

46. pointing device as claimed in claim 45, wherein beam condensing unit between light-emitting device and fingerprint deriving means,

The ratio of the distance between distance between light-emitting device and the beam condensing unit and beam condensing unit and the fingerprint deriving means is n: 1 and

N is from 1 to 5 real number.

47. as each the described pointing device in the claim 42 to 44, wherein CPU further comprises the function of handling the fingerprint recognition processing with software mode, and the fingerprint image that is used for will being stored in the fingerprint image of storage device and being received from the fingerprint deriving means compares.

48. pointing device as claimed in claim 42, wherein storage device is stored fingerprint image periodically or only store fingerprint image when storage device is received the request of CPU according to predetermined period.

49. one kind is used for providing service method to the portable terminal with fingerprint identification function, described method comprises:

First step obtains user's fingerprint image and the fingerprint image that the obtains fingerprint image with previous registration is compared by the pointing device with fingerprint identification function, to determine the identification to the user; With

Second step is according to the right to use of definite sorted users as a result of first step, to show corresponding to usufructuary menu.

50. method as claimed in claim 49 further comprises third step: make the user can use the required function of menu shown in second step.

51. method as claimed in claim 50, wherein when the identification carried out the user, second step is to show whole menu, and perhaps when the identification that not have to carry out to the user, second step is the certain menu that shows the permission use.

52. method as claimed in claim 51 wherein when the user who is not identified is intended to use other functions of menu permission use, shows that expression does not allow message of using and the menu that shows the permission use once more.

53. method as claimed in claim 49, wherein first step is carried out as background process, and with by using pointing device automatically to obtain fingerprint image when the user navigates with finger, and user's fingerprint is obtained in the processing that not have execution to add.