WO2006099682A1

WO2006099682A1 - Resource effective image processing system

Info

Publication number: WO2006099682A1
Application number: PCT/AU2006/000398
Authority: WO
Inventors: Jon Reid; Michael Soire; Maxim Shelevoy
Original assignee: Bio Recognition Systems Pty Ltd
Priority date: 2005-03-24
Filing date: 2006-03-24
Publication date: 2006-09-28

Abstract

The invention comprises a resource-effective image processing system comprising an image acquisition sensor system configured for acquiring a presented image; and a low power and resource microcontroller, configured in a parallel interface with the sensor and connected via wired or wireless support printed circuit board to a secure input/output device. The microcontroller is configured for testing the quality of the acquired image, adjusting the operational parameters of the image acquisition sensor system to improve the quality of the acquired image upon a consequent acquisition, saving the data of the entire image in memory facilitating comparison of the image with predetermined previously stored image, and forwarding at least one encrypted packet to at least one predetermined secure input/output device to trigger a predetermined action.

Description

RESOURCE EFFECTIVE IMAGE PROCESSING SYSTEM

TECHNICAL FIELD

The present invention relates to a system and method for image acquisition and processing. In particular, the invention refers to a resource-effective fingerprint image acquisition and processing system.

BACKGROUND OF THE INVENTION

Previous systems for fingerprint image acquisition are complex, costly and suffered many drawbacks in their use. These included:

• Complicated hardware development interfacing and software source code needed to be written for satisfactory integration, with the resultant substantial cost.

• To talk to and communicate with the devices, large complex libraries and control had to be established between the device and the controlling software. Additionally, overheads were required to process the large amounts of data from images and real time management of quality. • Other disadvantages included the need for a number of backend processes that the developer had to support in software for the integration to operate.

• The physical distance between device and host processor also caused problems. Typically the distance between the developers software and the location of the physical device has been limited to only a few metres.

An overview of resource intensive systems of the past include the following key areas of development and concern:

• Image Extraction • Image Control

• Image Gain and Quality Adjustment • Matching and Verification

• Speed of operations

• Integration with peripheral devices such as security controllers and third party devices • Memory management

SUMMARY OF THE INVENTION

According to one aspect of the invention, there is provided a resource-effective image processing system comprising; an image acquisition sensor system configured for acquiring a presented image; and a low power and resource microcontroller, configured in a parallel interface with the sensor and connected via wired or wireless support printed circuit board to a secure input/output device, wherein the microcontroller is configured for; testing the quality of the acquired image; adjusting the operational parameters of the image acquisition sensor system to improve the quality of the acquired image upon a consequent acquisition; saving the data of the entire image in memory; facilitating comparison of the image with predetermined previously stored image; and forwarding at least one encrypted packet to at least one predetermined secure input/output device to trigger a predetermined action.

Preferably, the microcontroller is programmed to internally compare the data associated with the image with the predetermined previously stored image.

Preferably, the image data is encrypted and sent to an external server for processing.

Preferably, the microcontroller is programmed to iteratively test the quality of the acquired image and adjust the operational parameters of the image acquisition sensor system until an image of a satisfactory quality is obtained. Preferably, the microcontroller is programmed to test the quality of the image by extracting a centre portion of the obtained image and counting the number of pixels for each of the grey scale pixel colours within this central portion, a complete image being extracted from the sensor and saved in memory only if the partial image is of a satisfactory quality.

Preferably, the microcontroller is programmed so as to adjust the operational parameters of the image acquisition sensor system if the number of black pixels is less than a number which will provide a count of 35% of black, 30% white and 35% combination of grey pixels.

Preferably, adjusting the operational parameters of the image acquisition sensor system comprises varying the automatic gain control and/or voltage levels of the image sensor.

Preferably, the microcontroller is programmed to save the complete image in memory by iteratively reading a portion of the image at a time, decoding and saving the portion into memory and appending it to any previously saved portions, until the entire image is saved, thus saving the amount of RAM required to fully buffer or store the image.

Preferably, the system further includes EPROM for storing permanent data associated with web pages and configuration data, as well as semipermanent volatile data, including image data, to reduce the RAM required, wherein at least a portion of the EPROM is located in programmable space of the microcontroller.

Preferably, the system further includes at least one of the following embedded applications:

DHCP client; HTTP server; BTP client; and

FTP client for communicating with other electronic devices in a network environment. Preferably, the system is configured to communicate with applications on external computer or network via software application programmable interface, wherein the communications between the software application programmable interface and the image processing system use a protocol transport layer.

Preferably, the system is configured so that communications from the software application programmable interface to the image processing system are sent over the network to a network interface hardware, forwarded to the TCP/IP stack and passed to the protocol layer.

Preferably, the HTTP server of the imaging system can be accessed by an external HTTP client to manually configure settings of the system.

Preferably, the DHCP client can be used for automatically changing an IP address of the system.

Preferably, the microcontroller is arranged to decrypt encrypted commands from external devices and encrypt status information before passing it to the BTP client for transport to an application programmable interface.

Preferably, the system is arranged for processing fingerprint images.

Preferably, the system further includes at least one of an LED and a buzzer for notifying the user of the system that a quality image is being obtained and/or what the outcome of the image comparison is.

Preferably, the imaging system further comprises a temper switch for resetting the system to its default values, the system being further configured to be able to inform an application programmable interface of temper switch activation and image extraction.

Preferably, the operational frequency of the microcontroller is less than 100 MHz. Even more preferably, the operational frequency of the microcontroller is less than 50 MHz.

Further preferably, the operational frequency of the microcontroller is 40 MHz or less.

Preferably, the system uses less than 2 Kb RAM.

Preferably, the system uses less than 5 Kb flash memory.

According to a second aspect of the invention there is provided a method for resource- effective image processing, the method comprising the steps of; acquiring a presented image; and testing the quality of the acquired image; adjusting the operational parameters of image acquisition to improve the quality of the acquired image upon a consequent acquisition; saving the data of the entire image in memory; comparing the image with predetermined previously stored image; and forwarding at least one encrypted packet to at least one predetermined secure input/output device to trigger a predetermined action.

Preferably, the quality of the image tested the operational parameters of image acquisition are adjusted until an image of a satisfactory quality is obtained.

Preferably, the quality of the image is tested by extracting a centre portion of the obtained image and counting the number of pixels for each of the grey scale pixel colours within this central portion, a complete image being extracted from the sensor and saved in memory only if the partial image is of a satisfactory quality.

Preferably, the image is adjusted if the number of black pixels is less than a number which will provide a count of 35% of black, 30% white and 35% combination of grey pixels. Preferably, adjusting the operational parameters of image acquisition includes varying the automatic gain control and/or voltage levels of the image acquisition system.

Preferably, the complete image is saved in memory by iteratively reading a portion of the image at a time, decoding and saving the portion into memory and by appending it to any previously saved portions, until the entire image is saved, thus saving the amount of RAM required to fully buffer or store the image.

Preferably, the required RAM for performing the method is reduced by saving permanent data associated with web pages and configuration data, as well as semipermanent volatile data, including image data, on EPROM.

Preferably, the method further includes the step of communicating with other electronic devices in a network environment.

Preferably, communication with applications on external computer or network are performed via software application programmable interface, wherein the communications between the software application programmable interface and the image processing system use a protocol transport layer.

Preferably, communications from the software application programmable interface to the image processing system are sent over the network to a network interface hardware that forwards them to the TCP/IP stack and passed to the protocol layer.

Preferably, the DHCP client can be used for automatically changing an IP address of the system. Preferably, the microcontroller is arranged to decrypt encrypted commands from external devices and encrypt status information before passing it to the BTP client for transport to the application programmable interface.

Preferably, the system is arranged for processing fingerprint images.

Preferably, the method further includes notifying a user of that a quality image is being obtained and/or what the outcome of the image comparison via a LED and/or a buzzer.

Preferably, the method further including the step of informing an application programmable interface of switch activations and image extraction.

Preferably, the processing is conducted at operational frequency of less than 100 MHz.

Even more preferably, the processing is conducted at operational frequency of less than 50 MHz.

Further preferably, the processing is conducted at operational frequency of less than 40 MHz or less.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 shows operation of the fingerprint image processing device of the invention in a network environment; Figure 2 shows a block diagram of the fingerprint image processing device of the invention.

DESCRIPTION

The system and the method of the invention are represented by a device that incorporates a new fingerprint recognition and image extraction hardware technology system that is universally applicable for identifying and verifying persons via their fingerprints. It is particularly useful because it utilises low resource and intelligent packetisation technology for use over remote media and this is cheap to produce.

The device comprises 4 parts; An outer casing or shell made of plastic cast aluminium or a mixture of alloys including zinc, magnesium and aluminium or a plastic injection compound containing a fingerprint recognition system,

Water resistant silicon gasket used to seal the device from weather and the environment, Metal bracket for attaching the device and locking the device to a surface and Main electronics and host printed circuit board

The device is preferably attached to a vertical or other surface by a method of four bayonet screws which securely attach an outer housing or casing onto a locking plate using key-like cut outs on the metal plate or bracket. Initially the device may be offered up slightly off centre to allow rotation of the casing and locking of the heads of the screws to the metal bracket. Rotation of the outer casing in respect of the metal bracket also locks down the casing onto the mounting bracket, providing a secure and tamper resistant connection.

The device is typically mounted on an exterior wall. Within the cavity of this outer housing is a single main printed circuit board on which is mounted all the sensor electronics, power supply, protective circuitry and control electronics. Optionally the design supports a stacked configuration whereby another single printed circuit board may be plugged into the main printed circuit board for optional communications standards, such as Ethernet, Weigand, Serial RS485/232, Wireless 802.1 lb/g and any other developed communications interfaces with which the unit be may required to operate.

The primary objective of this device is to allow extraction of fingerprint data for use by other systems remote from the location of the device at a relatively modest cost. It is primarily designed for security markets where physical access, document security, e- commerce and any other type of security may require a person to be identified.

An overview of the present invention below shows how little is required to operate and interact with our device from a development level:

• Wait for a message that an image has been extracted

• Process the image through the Matching

• Manage the result to the backend system

What in essence was done previously in a software environment the application has done at a hardware level, removing the time and cost away from third party software developers and ultimately reducing the cost to the end user. The applicant's device has been developed to universally service all persons willing to work with or wishing to own fingerprint technology for any purpose at low cost.

In the fingerprint recognition device of this invention, procedures, previously dealt with by remote software, are satisfied by hardware provided within the device. Since the device both acquires and processes the fingerprint image, by comparing it to other previously stored images, the device will be referred to as an image processing device.

The device has the ability to handle memory, images acquisition, image AGC (Automatic Gain Control), image quality adjustment, data encryption, tuning of images and allow all this to be done on the device hardware level rather than on remote software. It is a hardware intelligent image extraction and acquisition instrument that allows easy real world interfacing and makes available low cost devices for homeowners and any user wanting to be able to utilise the technology of fingerprint biometrics for their own use. The applicant's device can achieve this on low resources where previously high cost and high powered processors were required. Algorithms have been developed to run the same speed and functionality that is available on higher priced devices in a package that has been miniaturised and which costs less than other commercially available product of its kind.

A toolset has been developed which allows the integration of the device into any computing environment whether it be embedded or pc based. The applicant has developed a specialised code so that the number of instructions required for integration to the device are minimal. A developer can then utilise the device easily and quickly. AU previously handled overheads and backend algorithmic functions now are handled on hardware not software. This allows the much more efficient use of the available processing and storage resources. For example, the device can operate with a microcontroller with operational frequency of less than 100MHz, less than 50MHz, and even 40MHz. RAM of 1 to 2KB and flash memory of 2 to 5KB are also sufficient for the operation of the device.

The device of the invention comprises an image acquisition sensor and a low power and resource processor connected via wired or wireless support printed circuit board to a secure input/output device.

Referring now to the drawings, FIG.2 shows a block diagram of an imaging device, while FIG. 1 shows devices 102 and 103 connected to a LAN or Internet 121 via a wireless connection 100 such as 802.1 IG or a lObaseT connection 101.

Also connected to the LAN 121, is the software application programming interface, API 113, which communicates with the imaging devices using protocol transport layer. The software API 113 allows applications on the personal computer 114 to communicate to the imaging devices 102 or 103 without burdening the application with communication layers. Another scenario is to use the software API 106 to communicate to server 124 then to have applications on client computer 122, laptop 110 and PDA 107 communicating via the server 124. A HTTP client 116 can be used to access a HTTP server 207 in the image devices 102 and 103, to manually configure settings.

Image devices 102 or 103 requires its IP address to be changed, this can be done automatically using the DHCP (Dynamic Host Configuration Protocol) client 204 embedded in the device or can be configure manually using the HTTP server 207 embedded in the device. Further, the configuration data for the imaging device 102 can be reset to default values by activating the tamper switch 219 and touching the sensor 217 a programmed number of times within a programmed period of time. Configuration data such as IP addresses and serial numbers and other data are stored in EPROM 226 (Erasable Programmable Read-Only Memory). Once a valid IP address is obtained, communications to the API 113 or 106 can commence. This Allows the API 113 or 106 to control the LED indicators 218, the Sounder 216 and the Secure IO 220. The Secure IO 220 has relay and weigand IO functions. Further the Image device 102 can inform the API 113 of Tamper switch 219 activation and image extraction from sensor 217. Commands from the API 113 or 106 are sent over the LAN 121 to the imaging devices 102 network interface 200 processed by the TCP/IP stack 202 and passed to the BTP (Bio Transport Protocol) layer 205. BTP stands for Bio Transport Protocol which is protocol layer used to send commands and status information for the imaging device 102. Commands to the image devices 102 are decrypted by the encryption software in the microcontroller 215 and then processed by the microcontroller 215. Status information from the microcontroller 215 is sent to the encrypted by the microcontroller 215 and then passed to the BTP 205 layer for transport to the API 113 or 106.

The sensor 218 interfaces to the microcontroller 215 using a parallel data interface 224. This increases the efficiency in which the microcontroller 215 can extract information from the sensor 218. Thus removing any serialisation processes between the sensor 218 and the microcontroller 215.

On presentation of a finger onto the sensor 217 surface, the imaging device 102 drives the LED 218 and sounder 216 to confirm finger placement, then a centre portion of the image is extracted and tested for quality by the real time image optimiser RIO. Adjustments are made to the sensors 217 automatic gain control if they are required by the RIO to improve the image quality. The RIO works by counting the number of pixels for each of the eight gray scale pixel colours. A good image has an optimum ratio of black pixels to white pixels of about 30% white to 35% black with remainder being combination of gray. If the pixels ratios fall outside the optimum values, adjustments to the sensor gain and voltage levels are made. If adjustments are made to the sensor 217 another partial image is extracted and tested by the RIO. This process is repeated until a good quality partial image is extracted. Then a full image is extracted from the sensor 217 and saved in image storage EPROM 226 by the microcontroller 215. As there is limited RAM in the microcontroller 215, to extract out the full image, the microcontroller 215 reads one sixth of the image from the sensor 217 decodes the image data and saves the resulting portion of the image to EPROM 226. The next sixth of the image is then read from the sensor 217, decoded and appended to the portion previously saved in the EPROM 226, this is repeated until the complete image is saved to EPROM 226. This removes the need to have large amounts of RAM to fully buffer or store the image. The finger placement LED 218 is then altered and the buzzer 216 driven to inform the user they can remove their finger.

Use of EPROM 226 to both store permanent data such as web pages, configuration data and semi permanent volatile data such as the image eliminates additional RAM requirements. The EPROM 226 uses a two wire I²C interface 225 for data transfer to and from the microcontroller 215.

The microcontroller 215 assembles an encrypted image packet. This packet is then sent to the API 113. On reception of the encrypted packed, the API 113 decrypts the packet to extract the image. The image is matched against a database of images in the API 113.

If the match is successful the API 113 transmits a packet to the image device 102 to drive the LEDS 218 and sound the Buzzer 216 into a particular state to indicate success.

If the match was unsuccessful the API 113 transmits a different packet to the image device 102 to drive the LEDS 218 and sounds the Buzzer 216 into a different state to indicate matching failure. Further the API 113 may send an encrypted packet to the imaging device 102 that is passed along a interface 221 to one or more Secure IO devices 220 for the purpose of external device control via Relay, Weigand Security format, wireless devices, or any type of real world control

It should be appreciated that the disclosed apparatus and associated method are not limited to the particular embodiments described herein, but also cover other arrangements, using similar inventive concepts.

The terms "comprising" and "including" as used herein (and their grammatical variations) are used in the inclusive sense of "having" and not in the exclusive sense of "consisting only of.

Claims

1. Resource-effective image processing system comprising; an image acquisition sensor system configured for acquiring a presented image; and a low power and resource microcontroller, configured in a parallel interface with the sensor and connected via wired or wireless support printed circuit board to a secure input/output device, wherein the microcontroller is configured for; testing the quality of the acquired image; adjusting the operational parameters of the image acquisition sensor system to improve the quality of the acquired image upon a consequent acquisition; saving the data of the entire image in memory; facilitating comparison of the image with predetermined previously stored image; and forwarding at least one encrypted packet to at least one predetermined secure input/output device to trigger a predetermined action.

2. The system of claim 1 wherein the microcontroller is programmed to internally compare the data associated with the image with the predetermined previously stored image.

3. The system of claim 1, wherein the image data is encrypted and sent to an external server for processing.

4. The system of claim 1 wherein the microcontroller is programmed to iteratively test the quality of the acquired image and adjust the operational parameters of the image acquisition sensor system until an image of a satisfactory quality is obtained.

5. The system of claim 1 wherein the microcontroller is programmed to test the quality of the image by extracting a centre portion of the obtained image and counting the number of pixels for each of the grey scale pixel colours within this central portion, a complete image being extracted from the sensor and saved in memory only if the partial image is of a satisfactory quality.

6. The system of claim 1 wherein the microcontroller is programmed so as to adjust the operational parameters of the image acquisition sensor system if the number of black pixels is less than a number which will provide a count of 35% of black, 30% white and 35% combination of grey pixels.

7. The system of claim 1 wherein adjusting the operational parameters of the image acquisition sensor system comprises varying the automatic gain control and/or voltage levels of the image sensor.

8. The system of claim 1 wherein the microcontroller is programmed to save the complete image in memory by iteratively reading a portion of the image at a time, decoding and saving the portion into memory and appending it to any previously saved portions, until the entire image is saved, thus saving the amount of RAM required to fully buffer or store the image.

9. The system of claim 1, the system further includes EPROM for storing permanent data associated with web pages and configuration data, as well as semipermanent volatile data, including image data, to reduce the RAM required, wherein at least a portion of the EPROM is located in programmable space of the microcontroller.

10. The system of claim 1 , wherein the system further includes at least one of the following embedded applications:

DHCP client;

HTTP server;

BTP client; and

FTP client for communicating with other electronic devices in a network environment.

11. The system of claim 10₅ wherein the system is configured to communicate with applications on external computer or network via software application programmable interface, wherein the communications between the software application programmable interface and the image processing system use a protocol transport layer.

12. The system of claim 11 , wherein the system is configured so that communications from the software application programmable interface to the image processing system are sent over the network to a network interface hardware, forwarded to the TCP/IP stack and passed to the protocol layer.

13. The system of claim 10, wherein the HTTP server of the imaging system can be accessed by an external HTTP client to manually configure settings of the system.

14. The system of claim 10, wherein the DHCP client can be used for automatically changing an IP address of the system.

15. The system of claim 10, wherein the microcontroller is arranged to decrypt encrypted commands from external devices and encrypt status information before passing it to the BTP client for transport to an application programmable interface.

16. The system of claim 1 , wherein the system is arranged for processing fingerprint images.

17. The system of claim 16, wherein the system further includes at least one of an LED and a buzzer for notifying the user of the system that a quality image is being obtained and/or what the outcome of the image comparison is.

18. The system of claim 16, wherein the imaging system further comprises a temper switch for resetting the system to its default values, the system being further configured to be able to inform an application programmable interface of temper switch activation and image extraction.

19. The system of claim 1 , wherein the operational frequency of the microcontroller is less than 100 MHz.

20. The system of claim 1 , wherein the operational frequency of the microcontroller is less than 50 MHz.

21. The system of claim 1 , wherein the operational frequency of the microcontroller is 40 MHz or less.

22. The system of claim 1 , wherein the system uses less than 2 Kb RAM.

23. The system of claim 1, wherein the system uses less than 5 Kb flash memory.

24. A method for resource-effective image processing, the method comprising the steps of ; acquiring a presented image; and testing the quality of the acquired image; adjusting the operational parameters of image acquisition to improve the quality of the acquired image upon a consequent acquisition; saving the data of the entire image in memory; comparing the image with predetermined previously stored image; and forwarding at least one encrypted packet to at least one predetermined secure input/output device to trigger a predetermined action.

25. The method of claim 24 wherein the quality of the image tested the operational parameters of image acquisition are adjusted until an image of a satisfactory quality is obtained.

26. The method of claim 24 wherein the quality of the image is tested by extracting a centre portion of the obtained image and counting the number of pixels for each of the grey scale pixel colours within this central portion, a complete image being extracted from the sensor and saved in memory only if the partial image is of a satisfactory quality.

27. The method of claim 24, wherein the image is adjusted if the number of black pixels is less than a number which will provide a count of 35% of black, 30% white and 35% combination of grey pixels.

28. The method of claim 24 wherein adjusting the operational parameters of image acquisition includes varying the automatic gain control and/or voltage levels of the image acquisition system.

29. The method of claim 24 wherein the complete image is saved in memory by iteratively reading a portion of the image at a time, decoding and saving the portion into memory and by appending it to any previously saved portions, until the entire image is saved, thus saving the amount of RAM required to fully buffer or store the image.

30. The method of claim 24, wherein the required RAM for performing the method is reduced by saving permanent data associated with web pages and configuration data, as well as semipermanent volatile data, including image data, on EPROM.

31. The method of claim 24, wherein the method further includes the step of communicating with other electronic devices in a network environment.

32. The method of claim 31, wherein communication with applications on external computer or network are performed via software application programmable interface, wherein the communications between the software application programmable interface and the image processing system use a protocol transport layer.

33. The method of claim 34, wherein communications from the software application programmable interface to the image processing system are sent over the network to a network interface hardware that forwards them to the TCP/IP stack and passed to the protocol layer.

34. The method of claim 33 , wherein the HTTP server of the imaging system can be accessed by an external HTTP client to manually configure settings of the system.

35. The method of claim 33, wherein the DHCP client can be used for automatically changing an IP address of the system.

36. The method of claim 33, wherein the microcontroller is arranged to decrypt encrypted commands from external devices and encrypt status information before passing it to the BTP client for transport to the application programmable interface.

37. The method of claim 24, wherein the system is arranged for processing fingerprint images.

38. The method of claim 37, wherein the method further includes notifying a user of that a quality image is being obtained and/or what the outcome of the image comparison via a LED and/or a buzzer.

39. The method of claim 37, the method further including the step of informing an application programmable interface of switch activations and image extraction.

40. The method of claim 24, wherein the processing is conducted at operational frequency of less than 100 MHz.

41. The method of claim 24, wherein the processing is conducted at operational frequency of less than 50 MHz.

42. The method of claim 24, wherein the processing is conducted at operational frequency of less than 40 MHz or less.