KR20080045094A - Linear image sensing device with image matching function and processing method therefor - Google Patents

Abstract

A linear image sensing device with an image matching function and a processing method for the same are provided to miniaturize products and decrease consumption power consumption by performing a matching process through simple logic calculation without using a complex circuit. A linear sensor array(11) outputs the multiple fragment image analog signals by sensing the multiple overlapped fragment images. A programmable gain amplifier(11A) outputs the multiple amplified signals by amplifying the fragment image analog signals. An analog-digital converter(12) outputs the multiple digital image signals. An image matching module(13) including a navigation unit reproduces non-overlapped fragment images by matching the digital image signals, wherein the navigation unit outputs a navigation signal for controlling operations of a pointer system within a terminal device system. An input/output interface(14) is connected to the terminal device system, and outputs the non-overlapped fragment images to the terminal device system sequentially. A controlling logic(15) controls operations of the entire devices.

Description

Linear image sensing device having image matching function and processing method therefor {LINEAR IMAGE SENSING DEVICE WITH IMAGE MATCHING FUNCTION AND PROCESSING METHOD THEREFOR}

1 is a schematic diagram showing a conventional structure using a linear fingerprint sensor to read an image of a fingerprint.

2A to 2D illustrate examples of combining a plurality of fingerprint images into one complete fingerprint image.

3 is a schematic diagram of a linear image sensing device having an image matching function according to a first embodiment of the present invention.

4 is a schematic diagram of images obtained using the image sensing device of the present invention.

FIG. 5 is a schematic diagram illustrating image matching between a first fragment video signal and a second fragment video signal of FIG. 4.

FIG. 6 is a schematic diagram illustrating image matching between a second fragment video signal and a third fragment video signal of FIG. 4.

7 is a block diagram of a linear image sensing device having an image matching function according to a second embodiment of the present invention.

8 is a block diagram of a linear image sensing device having an image matching function according to a third embodiment of the present invention.

The present invention relates to a linear image sensing device having an image matching function and a processing method thereof, and more particularly, includes a memory buffer and image matching logic (ie, an algorithm), and continuously processes an entire inputted fingerprint image, It relates to a linear fingerprint sensing device capable of temporarily storing and outputting successive partial fragment images and then combining them into a complete fingerprint by stacking successive partial fragment images side by side. The full fingerprint image has the same size as the sensor area, and the partial fragment image has a size smaller than the sensor area. The invention also relates to the following patent applications by one of the inventors: (a) Taiwan Patent Application No. 091106806, filed April 3, 2002, titled the invention: "CAPACITIVE FINGERPRINT SENSOR) ", published in Taiwan Patent Publication No. TW583592, currently registered as Patent No. 200040; (b) Taiwan Patent Application No. 090112023, filed May 17, 2001, entitled “Capacitance Pressure Microsensor and Method for Manufacturing and Signal Detection Thereof”, currently registered as Invention Patent No. 182652; (c) Taiwan Patent Application No. 090113755, filed Jul. 6, 2001, titled: "THERMOELECTRIC SENSOR FOR FINGERPRINT THERMAL IMAGING", published in Taiwan Patent Publication No. TW558686, and present invention Registered as patent 189671; (d) Taiwan Patent Application No. 092113528, filed May 20, 2003, titled the invention: a sweep type fingerprint sensor module and a sensing method thereof; And Taiwan Patent Application No. 092114621, filed May 29, 2003, "Card Device with Sweep Fingerprint Sensor", currently registered as Invention Patent No. 225623.

Many kinds of fingerprint authentication techniques are known. The use of an ink pad and the direct movement of ink from the ink pad to the recording card by thumb or finger is a standard method of such confirmation. The optical scanner then scans the record card to obtain an image, which is compared with fingerprint images or templates in a computer database. However, the most serious disadvantage of the method described above is that fingerprint verification cannot be performed in real time, so that it cannot satisfy the requirements of real time authentication such as network authentication, e-business, portable electronic products, personal ID card, security system, and the like.

Reading fingerprints in real time has been an important issue in the biometrics market. Conventionally, optical fingerprint sensors have been used to read fingerprints in real time. However, optical fingerprint sensors have some disadvantages such as large size and high power consumption. Therefore, a silicon fingerprint sensor manufactured by silicon semiconductor technology has been developed that overcomes the disadvantages of the optical fingerprint sensor. For example, a capacitive fingerprint sensor sold by LIGHTUNING TECH. INC. With a product model number LTT-C500 has the above advantages.

Because of the size of the fingerprint, the sensing area of a conventional silicon fingerprint sensor is large, for example larger than 9 mm x 9 mm. Moreover, due to manufacturing limitations of silicon integrated circuits, only 50 to 70 good dies can be fabricated in 6 inch wires. Sensors are expensive in many applications. These expensive sensor prices limit the use of a variety of consumer electronic products such as silicon fingerprint sensors, notebook computers, mobile phones, PDAs, computer peripherals, or personal ID cards with embedded fingerprint sensors.

In order to overcome this cost problem, it is possible to reduce the one-dimensional length of the conventional two-dimensional face-type silicon fingerprint sensor to the one-dimensional length of the linear sensor structure in order to increase the number of excellent dies and lower the price of the sensing device. In this case, the sweep of the finger on the sensor surface and the entire finger are successively scanned into a plurality of full fragment images, which are then reconstructed into a complete image.

Maingue et al and Kramer, in US Pat. Nos. 6,289,114 and 6,317,508, disclose a method for reconstructing a linear fingerprint sensor and a plurality of superposed whole images into a complete image, as shown in FIGS. 1 and 2. have. 1 is a schematic diagram showing a conventional structure using a linear fingerprint sensor to read an image of a fingerprint. The sensor 110 is an array device in which the horizontal size is substantially equal to the width of the finger 120, the vertical size is smaller than the horizontal size, and the fingers are swept horizontally. Accordingly, the relative movement speed V of the finger 120 and the sensor 110 is generated. That is, finger 120 sweeps at a speed V over the surface of the sensor. Accordingly, the sensor 110 continuously acquires full fragment images, such as full fragment images 121a through 121s of FIG. 2A. Consecutive full fragment images 121a through 121s may be output by the microprocessor 130 having a data size as shown in FIG. 2B and then stored in a random access memory (RAM). Subsequently, the microprocessor 130 extracts consecutive fragment images 121a to 121s and constructs fingerprint images according to software logic stored in the read-only memory (ROM). First, images 121a and 121b are reconstructed into image 121ab, and then images 121c and 121ab are reconstructed into image 121abc as shown in FIG. 2C. This process is repeated in an analog manner so that the fragment images 121a through 121s are reconstructed into a complete fingerprint image 122 corresponding to the fingerprint as shown in FIG. 2D.

This method can obtain a relatively large fingerprint image without the use of a large area sensor, which is advantageous in terms of cost savings and low false access rate and low false rejection rate similar to that obtainable by a large area fingerprint sensor. It is advantageous in terms of improvement of recognition quality such as false access rate.

However, the structure and method of the sensor 110 has some disadvantages. First, hundreds of fragment images must be acquired within a very short period of time (less than 1 second). For example, if the finger sweep speed is 10 cm / sec and the fingerprint sensor specification is 8 × 280 (ie “Atmel Fingerchip” specification, 500 dpi), then the RAM 140 must have a capacity greater than 600 Kbytes and more. Since large buffer memory is required for subsequent reconstruction, the price of the system rises. The '114 patent forms a second combined image by combining the first combined image formed by combining the first fragment image with the second fragment image with the third fingerprint image. The second combined image is then combined with the fourth fingerprint image to form a third combined image. In this case, the memory occupied by the combined image gradually increases so that the buffer memory must be large enough to allow all fingerprint images to be combined.

Furthermore, in order to complete the fingerprint recognition process within one second (typically an acceptable period of less than two seconds) after sweeping the finger on the chip surface, between the chip of the sensor 110 and the microprocessor 130 of the terminal system. The communication interface must be an interface such as a parallel interface having a large bandwidth DMA mode or an express serial interface of USB2.0. Thus, a typical I ² C or low speed SPI or RS232 interface cannot be employed for transmission and design flexibility is limited. Because the microprocessor must be very fast, the microprocessor must be a DSP, for example.

Ericson disclosed a fingerprint sensing device including a memory buffer in US Patent Publication No. 2003/0021495. The advantage of the '495 patent is that the image transmission of the microprocessor of the terminal system is very flexible. However, the problem of transmission of image data through a broadband interface in a very short period (less than 1 second) of RAM of the terminal system is still not solved. Since the speed of operation of the microprocessor is very fast, the microprocessor must be a DSP, for example. In addition, the '495 patent does not mention how to solve the problem in the subsequent image processing method.

Accordingly, one object of the present invention is to provide a linear image sensing device having an image matching function and a processing method thereof, wherein the image sensing device has a data size output by the image sensing device, a data transmission bandwidth between a chip and a terminal system. And a plurality of non-overlapped partial fragment images in order to significantly reduce the memory capacity of the image sensing device and the terminal system CPU speed.

The present invention achieves the above object by providing an image sensing device electrically connected to the terminal system. The image sensing device includes a sensor array, a programmable gain amplifier, an analog-to-digital converter, an image matching module, an input / output interface, and control logic. The sensor array detects a plurality of overlapped whole fragment images of the object as the object moves over the sensor array along the direction of movement, thereby providing a plurality of whole fragment image anlog signals. Outputs The programmable gain amplifier amplifies the entire fractional video analog signal and outputs a plurality of amplified signals. The analog-digital converter receives the sequentially amplified signal and converts the amplified signal into a plurality of digital video signals. The image matching module receives and matches digital image signals to sequentially reproduce non-overlapped partial fragment images that are not overlapped with each other. An input / output interface electrically connected to the terminal system in a wired (direct) or wireless manner sequentially outputs non-overlapping partial fragment images to the terminal system. Control logic controls the operation of the sensor array, programmable gain amplifiers, analog-to-digital converters, image matching modules, and input / output interfaces.

The present invention also achieves the above object by providing a processing method of the image sensing device. The method includes the following steps: using a sensor array to detect a plurality of overlapping full fragment images of the object as the object moves along the direction of movement to obtain a plurality of fragment image analog signals; Amplifying the fragment image analog signal and outputting a plurality of amplified signals; Converting the sequentially amplified signal into a plurality of digital video signals; And matching the digital image signals to reproduce non-overlapping partial fragment images that are not sequentially overlapped with each other.

3 is a block diagram of a linear image sensing device having an image matching function according to a first embodiment of the present invention. 4 is a schematic diagram of images obtained using the image sensing device of the present invention. FIG. 5 is a schematic diagram illustrating image matching between a first fragment image signal and a second fragment image signal of FIG. 4. FIG. 6 is a schematic diagram illustrating image matching between a second fragment image signal and a third fragment image signal of FIG. 4.

3 to 6, the linear image sensing device 10 of the present embodiment is electrically connected to a terminal system 40 such as a computer, a mobile phone, a PDA, or the like. The linear image sensing device 10 includes a sensor array 11, a programmable gain amplifier 11A, an analog-to-digital converter 12, an image matching module 13, an input / output (I / O) interface 14, and Control logic 15.

The sensor array 11 is disposed on the substrate. In this embodiment, the substrate is made of a semiconductor, such as a semiconductor material. In other embodiments, such substrates may be made of an insulating material such as glass, polymer or other known material. The sensor may be a capacitive sensor, a pressure sensor, a temperature sensor, an optical sensor, an electric field sensor, or a magnetic field sensor, a sensor as disclosed in the above-mentioned (a) to (c) patent applications, and other types of sensors.

In this embodiment, the sensor is a capacitive sensor and the sensor array has a specification of 168 × 8 sensing member pixels. Each sensing member pixel includes a sensing electrode and a sensing circuit on a substrate below the sensing member pixel. Each pixel has a size of 50x50 microns and a resolution of 508 DPI.

The linear image sensing device 10 detects a plurality of overlapping full fragment images of a finger (object) when the finger moves sequentially along the direction of movement on the sensor array 11, thereby providing a plurality of full fragment image analog signals ( AFS). Programmable gain amplifier 11A properly amplifies the entire fractional video analog signal AFS and outputs the amplified signal AFSA. The analog-to-digital converter 12 receives the sequentially amplified signal AFSA and converts it into a plurality of digital image signals DFS. For the sake of simplicity, the digital image signal DFS sequentially receives the first full fragment image signal DFS1, the second full fragment image signal DFS2 and the third full fragment image signal DFS3 as shown in FIG. Include. The image matching module 13 receives and matches the digital image signal DFS, and sequentially reproduces the plurality of non-overlapping partial fragment images RFS. The fragment image (RFS) sequentially includes a first partial fragment image (RFS1), a second partial fragment image (RFS2), and a third partial fragment image (RFS3). The images (RFS1 to RFS3) usually have different lengths, but may have the same length under special conditions where the finger speed is always constant. The input / output interface 14 is electrically connected to the terminal system 40, and sequentially outputs non-overlapping fragment images (RFS) to the terminal system 40. Since the plurality of non-overlapping fragment images RFS do not overlap each other, the number of pixels of each fragment image RFS in the moving direction is less than or equal to the number of sensing units of the sensor array 11 in the moving direction. In the present invention, the entire fingerprint image has the same size as that of the sensor area and the partial fragment image has a size smaller than the size of the sensor area.

When the third fragment image RFS3 is the last output signal, since the non-overlapping fragment images RFS do not overlap each other, the data size of the first fragment image RFS1 or the second fragment image RFS2 is set to 3rd. It is less than or equal to the data size of the fragment image (RFS3).

The control logic 15 controls the operation of the sensor array 11, the programmable gain amplifier 11A, the analog-to-digital converter 12, the image matching module 13 and the input / output interface 14. The image matching module 13 receives and matches the first fragment image signal DFS1 and the second fragment image signal DFS2 and outputs a matching result. In this case, the first fragment image RFS1 is a non-overlapping image obtained by subtracting an overlapping portion between the second fragment image signal DFS2 and the first fragment image signal DFS1 from the first fragment image signal DFS1. In order to reduce the chip area, cost and power consumption of the chip device of the present invention, the operation logic (i.e. algorithm) used in the image matching module, which is the integrated circuit in the present invention, is very simple and floating point as in a microprocessor or DSP. It should not require structures such as floating point calculations. Instead, typical digital logic circuits can handle this process to reduce area, improve speed, and save power. The image matching logic used in this embodiment is to match the intensity distributions of the two fragment images based on the adjacent least-square method. It is also possible to measure the overlapped area of two neighboring fragment images by measuring the period for acquiring the fragment image.

In order to obtain the image matching function, the image matching module 13 includes a memory 31, an image matching unit 32 and a memory control unit 33. The memory 31 has two adjacent ones of the digital image signals DFS at one time (for example, the first fragment image signal DFS1 and the second fragment image signal DFS2 in FIG. 5 or the second one in FIG. 6). The second fragment image signal DFS2 and the third fragment image signal DFS3 may be temporarily stored. The image matching unit 32 matches two adjacent digital image signals DFS stored in the memory 31 and reproduces the non-overlapping fragment images RFS according to the above-described mathematical logic. The memory control unit 33 controls the operations of the memory 31 and the image matching unit 32.

The image sensing device of the present invention may provide a processing method including the following steps.

First, multiple overlapping fractional images of a finger are detected when an object (eg, a finger) moves to obtain a plurality of fragment image analog signals (AFS). The fragment image analog signals AFS are then properly amplified and the amplified signals AFSA are output. Next, the amplified signals AFSA are sequentially converted into a plurality of digital image signals DFS, which are sequentially the first fragment image signal DFS1, the second fragment image signal DFS2, and the third fragment. Video signal DFS3. Subsequently, a plurality of non-overlapping fragment images RFS are reproduced by matching two adjacent ones of the digital image signals DFS, wherein the non-overlapping fragment images are the first fragment image RFS1 and the second fragment image. (RFS2) and third fragment image (RFS3). Each of these consecutive, non-overlapping fragment images (RFS) can be processed by the memory control unit 33 and temporarily stored in the buffer 31C. Subsequently, the control logic 15 controls the non-overlapping fragment images (RFS) and outputs them to the terminal system 40 through the I / O interface 14, and then the non-overlapping fragment images are stacked in a manner of stacking the images side by side. Combined into a complete fingerprint image.

Hereinafter, a matching method of the digital image signal DFS will be described. The buffers 31A and 31B of the memory 31 receive and store the first fragment image signal DFS1 and the second fragment image signal DFS2, respectively, as shown in FIGS. 5 and 3. The image matching unit 32 then matches the first fragment image signal DFS1 with the second fragment image signal DFS2 to obtain a first overlap signal OS1. Next, the first overlapped signal OS1 is subtracted from the first fragment image signal DFS1, and the first fragment image RFS1 is obtained and output. Therefore, the first fragment image signal RFS1 is obtained by subtracting the superimposition signal OS1 between the first fragment image signal DFS1 and the second fragment image signal DFS2 from the first fragment image signal DFS1. Next, the control logic 15 outputs the first fragment image RFS1 to the memory buffer 31C and then outputs the terminal fragment 40 to the terminal system 40 through the I / O interface 14.

6 and 3, the first fragment image signal DFS1 stored in the memory buffer 31A is erased. Next, an empty buffer (eg, 31A) of the memory 31 receives and stores the third fragment image signal DFS3. Next, the second fragment image signal DFS2 and the third fragment image signal DFS3 are matched to obtain a second overlap signal OS2. Subsequently, the second fragment image RFS2 is obtained by subtracting the second superimposition signal OS2 from the second fragment image signal DFS2. Finally, a point is added to the third fragment image signal DFS3 or a point is removed from the third fragment image signal to generate and output a third fragment image RFS3. It should be noted that the present invention is not limited to the embodiment of the three fragment image signal and the three fragment images. Instead, the present invention is also suitable for applications with two or three or more fragment image signals and fragment images. Since the related processing steps are similar to each other, description thereof will be omitted.

During the matching process, the third fragment image signal DFS3 is shifted to the right with respect to the first fragment image signal DFS1 according to the sum of the signals DFS1 and DFS2 and the signals DFS2 and DFS3. Accordingly, the second fragment image RFS2 includes the left block 55 but does not include the right block 56 and the third fragment image RFS3 includes the left block 57 but the right block 58. Does not include

In order to efficiently manage and reduce the capacity of the memory 31, the speeds of the series of operations including the memory writing speed, the processing speed of the image matching unit, and the output speed of the non-overlapping fragment images should be much faster than the finger movement speed. For example, the duration for the sequence of operations must be shorter than one millisecond.

Thus, the memory 31 should only store two fractional signals, and the minimum capacity of the memory 31 should be substantially equal to twice the size of the data of each digital video signal DFS.

Therefore, the chip device of the present invention can continuously output non-overlapping fragment images according to the image matching unit and the memory control method. When the finger is positioned and moved over the chip device, the motion information on the X-axis and Y-sides changes the length and the change in length or the change in width or the change in width of blocks 57 and 58 similar to the movement of the finger on the digital panel. It can be provided by measuring the output format of consecutive, non-overlapping fragment images such as.

7 is a block diagram of a linear image sensing device having an image matching function according to a second embodiment of the present invention. As shown in FIG. 7, this embodiment is similar to the first embodiment except that the image matching module 13 includes a navigation unit 34 controlled by the control logic 15. The navigation unit 34 outputs the navigation signal NS through the input / output interface 14 to control the operation of the pointer system in the terminal system 40 so that the non-overlapping fragment images (RFS) and the digital image signal can be controlled. According to the DFS, the relative motion between two adjacent ones of the digital image signals DFS is calculated.

It should be noted that the linear image sensing device of this embodiment can provide the navigation signal (NS) output function without outputting only the fragment image.

8 is a block diagram of a linear image sensing device having an image matching function according to a third embodiment of the present invention. As shown in Fig. 8, the third embodiment is similar to the second embodiment except that the third embodiment does not output non-overlapping fragment images. Thus, the sensor array 11 detects a plurality of overlapping fractional images of a finger when the finger moves in any direction on the array to obtain a plurality of fractional image analog signals (AFS). Programmable gain amplifier 11A and analog-to-digital converter 12 have the same functions as those of the second embodiment, and thus detailed description thereof is omitted. The image matching module 13 outputs the navigation signal NS through the input / output interface 14 to receive and match the digital image signal DFS so as to control the operation of the pointer system in the terminal system 40. The relative motion between two adjacent ones of the digital image signals DFS is calculated.

The image matching module 13 includes a memory 31, a navigation unit 34, and a memory control unit 33. The memory 31 temporarily stores two adjacent ones of the digital image signals at one time. The navigation unit 34 generates a navigation signal NS by matching two adjacent signals among the digital image signals DFS stored in the memory 31. The memory control unit 33 controls the operations of the memory 31 and the navigation unit 34.

The processing method for the linear image sensing device according to the third embodiment of the present invention includes the following steps. First, the sensor array 11 detects a plurality of overlapping fragmented images of the finger when the finger moves in an arbitrary direction on the array to obtain a plurality of fragmented image analog signals (AFS). Subsequently, the fragment image analog signals AFS are amplified to output a plurality of amplified signals AFSA. The amplified signals AFSA are sequentially converted into a plurality of digital image signals DFS. The digital image signals DFS are then compared and matched such that relative motion between two adjacent ones of the digital image signals DFS is calculated. Therefore, the navigation signal NS is output to control the operation of the pointer system in the terminal system 40.

According to an embodiment of the present invention, the linear image sensing device performs a matching process by simple logic calculation without using a complicated processing circuit. Thus, power-saving and product miniaturization effects can be obtained. In addition, since the minimum capacity of the memory can be equal to twice the size of the data of the digital video signal, the cost and power consumption of the sensing device can be reduced. In addition, since the data size at each transmission is not larger than the data size of the digital video signal, overlapping does not occur, and the bandwidth between the sensing device and the terminal system of the present invention can be lowered and the memory space in the terminal system can be greatly reduced.

While the invention has been described above in terms of embodiments and preferred embodiments, it should be understood that the invention is not limited to these examples. On the contrary, the invention is intended to cover various modifications. Therefore, the scope of the appended claims should be construed broadly to encompass all such modifications.

Claims (7)

Linear sensor array that detects multiple overlapping full-fragment images of an object and outputs a plurality of fragment image analog signals when the object moves sequentially over the sensor array along the direction of movement ; A programmable gain amplifier for amplifying fractional video analog signals and outputting a plurality of amplified signals; An analog-to-digital converter that sequentially receives and converts the amplified signals and outputs a plurality of digital image signals; An image matching module for receiving and matching digital image signals to sequentially reproduce non-overlapping partial fragment images that are not superimposed on each other, wherein the image matching module includes non-overlapping partial fragment images and digital image signals. An image matching module including a navigation unit for calculating a relative movement relationship between two adjacent ones and outputting a navigation signal for controlling an operation of a pointer system in a terminal system through an input / output interface; An input / output interface electrically connected directly or indirectly to the terminal system and sequentially outputting non-overlapping partial fragment images to the terminal system; And An image sensing device electrically connected to a terminal system comprising a sensor array, a programmable gain amplifier, an analog-to-digital converter, an image matching module and control logic to control the operation of the input / output interface. 2. An image sensing device electrically connected to a terminal system according to claim 1, wherein the number of pixels of each non-overlapping partial fragment images in the movement direction is less than or equal to the number of sensing units of the sensor array in the movement direction. . The non-overlapping partial fragment images of claim 1, wherein the non-overlapping partial fragment images sequentially include a first fragment image, a second fragment image, and a third fragment image, wherein any one of the first fragment image and the second fragment image is a third fragment image. An image sensing device electrically connected to a terminal system, characterized in that the data size is less than or equal to the size of the image data. The method of claim 1, wherein the image matching module is A memory for temporarily storing two adjacent ones of the digital image signals; An image matching unit for generating non-overlapping partial fragment images by matching two adjacent ones of the digital image signals stored in the memory; And And a memory control unit which controls the operation of the memory and the image matching unit. 5. The image sensing device of claim 4, wherein the capacity of the memory is substantially equal to twice the size of the data of the digital image signal. 2. The method of claim 1, wherein the digital video signal sequentially comprises a first fragment video signal and a second fragment video signal, wherein the non-overlapping partial fragment images are derived from the first fragment image signal and the second fragment image signal. And a first fragment image corresponding to the superimposition signal obtained by subtracting one of the image signals. Detecting a plurality of overlapping full fragment images using a sensor array to obtain a plurality of fragment image analog signals when the object moves sequentially along the direction of movement; Amplifying the fragment image analog signals and outputting a plurality of amplified signals; Converting the sequentially amplified signals into a plurality of digital image signals; And Matching the digital image signals to sequentially reproduce non-overlapping partial fragment images not overlapping each other; And Calculating a relative movement relationship between two adjacent ones of the digital image signals according to the non-overlapping partial fragment images and the digital image signals, and outputting a navigation signal for controlling the operation of the pointer system in the terminal system. A processing method of an image sensing device.

Images