TWI712005B - Multi-spectrum high-precision object identification method - Google Patents

Abstract

A method for multi-spectrum and high-precision object identification, mainly using a multi-spectrum light-emitting unit to emit light of different frequencies to project the object under test, and a multi-spectrum image sensing unit to capture the image reflected by the object under different frequencies , Taking X-axis and Y-axis as a single plane image, and taking different spectral wavelengths of Z-axis as shadow depth. The sampling wavelength of Z-axis includes at least two infrared light narrow-range image signals, and the spectral range is distributed between 850nm and 1050nm. The frequency band of each infrared light is at least between 10nm and 60nm. The multiple X-axis and Y-axis single plane images sampled in different bandwidth bands on the Z axis are calculated and superimposed into a 3D relief image. For accurate comparison and identification, it can be widely used in security monitoring, industrial monitoring, face recognition, vehicle image recognition and door opening, etc.

Description

Multi-spectrum high-precision object identification method

The present invention relates to the technical field of image recognition, in particular to a method for multi-spectrum and high-precision object recognition. The specific operation can simply and effectively superimpose to produce a high-quality 3D relief image for accurate comparison and recognition.

Image sensors mainly use photoelectric conversion to obtain flat images. With the advancement of technology, they have been widely used in security monitoring, industrial monitoring, face recognition equipment, network cameras, drones, robots, and car reversing auxiliary image shooting. Products.

In particular, because face recognition has many advantages such as natural, simple, and non-contact in use, it can achieve the purpose of recognition without interfering with people’s normal behavior. This is even more important in today’s era when Internet-developed smart mobile devices are popular. In recent years, it has achieved rapid development, especially in the fields of identity recognition, information security control, financial payment, medical applications, and visual monitoring.

At present, the more advanced face recognition mainly uses the following two 3D stereoscopic image sensing technologies to achieve:

1. Time of Flight (ToF: Time of Flight): Use an infrared light source to illuminate the surface of the object and reflect it back. Since the speed of light (v) is known, an infrared light image sensor can be used to measure the object at different depths and reflect it back Time (t), the distance (depth) of different positions of the object can be calculated using simple mathematical formulas.

2. Structured light: Use special light sources to produce different light patterns, which will be reflected back from different depths of the object to cause the light patterns to be distorted for identification. For example, the current advanced Apple iPhoneX smartphones use dot matrix projectors ( Dot projector, which uses a high-power vertical cavity surface-emitting laser to emit infrared light lasers, which are produced by wafer-level optics (Wafer Level Optics, WLO), Diffractive Optical Elements (DOE) and other structures About 30,000 "Structured" light points are projected onto the user's face, and the array formed by these light points is reflected back to the Infrared camera to calculate the distance (depth) of different positions on the face.

As shown in Figure 10, in order to more effectively combine the above two methods to improve the accuracy of facial recognition, the current advanced Apple iPhoneX smartphones must be equipped with an infrared lens a1 and a 7-megapixel lens a2. , A flood illuminator a3 (Flood illuminator), a proximity sensor a4 (Proximity sensor), an ambient light sensor a5 (Ambient light sensor) and a dot array projector a6 (Dot projector) can be achieved. The applied high-precision combination not only has many components, high cost, but also takes up space for assembly.

More importantly, even though such high-unit-price precision components have been used to produce about 30,000 "Structured" light points projected on the user's face, in fact, the recognition effect obtained is still vulnerable to external The impact of changes in ambient light makes significant changes when capturing facial feature information, which leads to errors in subsequent facial information comparisons, and reduces the accuracy of facial feature comparisons, which greatly affects face recognition. performance.

Unfortunately, these shortcomings not only occur in smart phone applications, but also in other face recognition related products that apply the same technology, but they have not been improved and effectively overcome.

In view of this, the main purpose of the present invention is to provide a method for multi-spectrum high-precision object identification, which mainly includes the following steps:

Build an identification hardware mechanism in an identification system, which has at least one multi-spectrum light-emitting unit and one multi-spectrum image sensing unit;

The multi-spectrum light-emitting unit emits light of different frequencies to project the object to be measured, and the light emitted by the multi-spectrum light-emitting unit includes at least two infrared rays, and the spectral range is distributed between 850 nm and 1050 nm;

The multi-spectral image sensing unit is used to capture images projected by the object under different frequencies. The multi-spectral image sensing unit captures a narrow-area image signal including at least two infrared light reflections. The corresponding interval distribution of the spectrum light-emitting unit is between 850nm and 1050nm, and the narrow-band image signal wavelength bandwidth of each infrared light is between 10nm and 60nm;

Take X-axis and Y-axis as a single plane image, and use different spectral wavelengths of Z-axis as shadow depth. The sampling wavelength of Z-axis includes at least two infrared light narrow-range image signals, and the spectral range is similar to that of the multi-spectrum image sensing unit. Corresponding to images with an interval distribution between 850nm and 1050nm, and the frequency band of each infrared light is at least between 10nm and 60nm;

The multiple X-axis and Y-axis single plane images sampled in different bandwidth bands of the Z axis are calculated and superimposed into a 3D relief image for accurate comparison and identification.

As a result, the present invention can be widely used in security monitoring, industrial monitoring, face recognition, vehicle image recognition and door opening, etc., especially when applied to smart mobile devices, with fewer components, which can greatly reduce costs and save space. In addition, 3D relief images can be obtained quickly and accurately in use, and it can be significantly improved and less affected by ambient light, so it can effectively improve the accuracy of the entire face recognition, which is the biggest feature of the present invention.

Compared with the existing conventional technology, the conventional technology uses high unit price and precise components to produce structured light with special effects to project on the object to be measured, but because it cannot be technically broken, it can only use ordinary image sensing units. It is of course easier to be affected by external ambient light in the process, and the received image is not affected by ambient light. Even if structured light is used with time-of-flight distance measurement to generate 3D relief images, the overall recognition effect is of course It will also become worse, so that the accuracy of identification is significantly reduced.

On the other hand, the present invention uses a low-unit-price ordinary multi-spectrum light-emitting unit to project the light on the object to be measured with a general flood light source, and then uses a high-precision multi-spectrum image sensing unit with clear stereo imaging of the front and rear levels to receive it. The front and back levels of the image are clear and less affected by ambient light, and the multiple X-axis and Y-axis single plane images sampled in different bandwidth bands on the Z-axis are then superimposed to naturally obtain an accurate 3D relief image for use. Comparison recognition, therefore, the recognition effect of biological or non-biological entities must be better, and it is much simpler and more accurate than the prior art.

In order to facilitate the further introduction and disclosure of the purpose, composition method, application functional characteristics and effects of the present invention, the embodiments are combined with the drawings, and the detailed description is as follows: As shown in Figures 1 to 5, the present invention sets A method for multi-spectrum and high-precision object identification, mainly includes the following steps:

Build a recognition hardware mechanism 1 in a recognition system 100, which has at least a multi-spectrum light-emitting unit 2 and a multi-spectrum image sensing unit 3;

The multi-spectrum light-emitting unit 2 emits light of different frequencies to project the object 90 to be measured, and the light emitted by the multi-spectrum light-emitting unit 2 includes at least two infrared rays, and the spectral range interval distribution is between 850 nm and 1050 nm;

The multi-spectral image sensing unit 3 captures images projected by the object 90 under different frequencies of light, and the multi-spectral image sensing unit 3 captures narrow-area image signals 301 and 302 including at least two infrared light reflections, Its spectral range and the corresponding interval distribution of the multi-spectrum light-emitting unit 2 are between 850 nm and 1050 nm, and the narrow-band image signals 301 and 302 of each infrared light have a wavelength bandwidth between 10 nm and 60 nm;

Take X-axis and Y-axis as a single plane image 4, and use the different spectral wavelengths of Z-axis as shadow depth. The sampling wavelength of Z-axis includes at least two infrared light narrow-range image signals 301 and 302, the spectral range and the multi-spectral image The sensing unit 3 corresponds to images with an interval distribution between 850nm and 1050nm, and the narrow-band image signals 301 and 302 of each infrared light have a frequency band between at least 10nm and 60nm;

As shown in Figure 2, the multiple X-axis and Y-axis single plane images 4 sampled in different bandwidth bands on the Z axis are calculated and superimposed into a 3D relief image 5 for accurate comparison and identification.

In a preferred implementation, as shown in Figure 3, the multi-spectrum light-emitting unit 2 can be composed of a plurality of different-frequency light-emitting diodes 21 or a single multi-frequency light-emitting diode 20, and the single light-emitting diode can be multi-frequency. The light-emitting diode 20 can emit at least two kinds of infrared light in the range of 850nm to 1050nm. Preferably, the two infrared lights can be 850nm and 940nm, or 940nm and 1050nm, but they are not actually limit.

Preferably, as shown in Figures 3 to 5, the image sensing unit 3 is composed of a plurality of image sensors 31 of different frequency spectra or a single multi-frequency image sensor 30, and the single multi-frequency image sensor The image sensor 30 mainly includes: a photosensitive image element array 310 and its connected packaging circuit 311 to drive and control the photosensitive image element array 310 to capture external light and convert it into an output combined image signal, wherein the photosensitive image element array 310 It can capture RGB full-color visible light and IR infrared invisible light for photoelectric conversion, and; an image enhancement processing unit 312, built in the package circuit 311, to control the image captured by the photosensitive pixel array 310, including: The wide-area image signal 305 of color RGB visible light, the wavelength spectrum range of the full-color RGB visible light is between 400nm and 700nm, and the narrow-area image signal 301, 302 of at least two infrared non-visible light, whose range interval is distributed from 850nm to 700nm. Between 940nm, the wavelength bandwidth of each infrared non-visible narrow-area image signal 301 or 302 is between 10nm and 60nm, and the one wide-area image signal 305 and the two narrow-area image signals 301 and 302 are re-integrated Stack to form a clear output image with a three-dimensional sense of front and back levels.

Among them, because the combined image output signal is through the two narrow-band image signals 301, 302 stacked with different spectral ranges of infrared light waves distributed between 850nm and 1050nm, the recognition effect is far better than the conventional structure, and the level of stereoscopic presentation is improved. And clarity, and can indeed achieve the practical purpose of using a single multi-frequency image sensor 30 to capture images clearly and less affected by changes in ambient light for image recognition.

The preferred implementation is an application example of the present invention, in which the test object 90 can be the face of a human body, for example, it is applied to a mobile device that is currently popular for facial recognition power-on, or a facial recognition automatic payment mechanism...etc.

In a preferred implementation, as shown in Figure 6, the identification method for the specific application above can be further provided with a preliminary identification learning unit 6. For example, the two infrared rays of the multi-spectrum light-emitting unit 2 have been basically selected to be 850 nm and 940 nm, respectively, S101 Take the two infrared spectra 850nm and 940nm as the reference to take at least one image at the top, bottom, middle, left, and right angles of an original 60, that is, take a total of two images under different infrared spectra of 850nm and 940nm, and S102 as the original In the cross-transposition movement of the object, at least one image is taken at each interval angle, that is, at different interval angles, a total of two images are taken based on the infrared spectrum at 850nm and 940nm. S103 uses the Z axis to have different bandwidths and its narrow range The multiple X-axis and Y-axis single plane images 40 sampled in the 850nm and 940nm wavelength bands of the infrared spectrum are calculated and combined to form a reference three-dimensional relief image 65 of the original 60 for subsequent comparison and identification.

In a preferred implementation, the preliminary recognition learning unit 6 may further emit intermittent sounds or voices when it is executed, so that it can be conveniently used as a reference indication for the corresponding up, middle, down, left and right angular displacement speed of the original 60.

Preferably, after the above-mentioned reference three-dimensional relief image 65 of the original object 60 is filed, S201, when the system performs identification on the test object 90 to obtain the 3D relief image 5, first S202 will determine whether the test object 90 is If a biological or non-biological concrete entity is judged to be a concrete entity, S203 will be further compared with the reference three-dimensional relief image 65 of the original 60 stored in the preliminary identification learning unit 6, and S204 will be executed if the comparison is correct. If it is not correct, it will not be activated. For example, it can be used for automatic activation of mobile phone face recognition or agree to pay for activation, etc., or other different application areas that automatically activate using face recognition.

In a preferred implementation, as shown in Figure 7, an ambient light sensor 70 and an ambient light enhancement comparison unit 7 can be further provided on the identification system 100, wherein when the ambient light sensor 70 detects the ambient light When the first dark level is reached, the ambient light enhancement comparison unit 7 is activated to compare the 3D relief image 5 of the test object 90 with the reference three-dimensional relief image 65 of the original 60 obtained by infrared light at 940nm, and when the environment The light sensor 70 detects that the ambient light reaches the second darker level. At this time, the ambient light enhancement comparison unit 7 automatically switches to obtain the reference stereo of the original 60 with the 3D relief image 5 of the test object 90 and the infrared light at 850nm The embossed image 65 is compared, so that it can be adjusted according to different ambient light brightness to obtain a more accurate image recognition effect.

Preferably, the identification hardware mechanism 1 is set on a smart mobile device, such as a smart phone, a tablet computer, etc., but it is not limited to this, and can actually be set on a desktop computer or notebook Type computer. As shown in Figure 8, if the face recognition of the present invention is applied to a smart phone, the structure basically only needs to use a single multi-frequency light emitting diode 20 and a single single multi-frequency image sensor 30 With the addition of an ambient light sensor 70 to achieve this, the entire configuration is definitely more cost-effective and space-saving than the existing design of the Apple iPhone X mobile phone, and more importantly, the overall recognition accuracy must be more effectively improved.

Even, the preferred implementation, as shown in Figure 3, if more precise is required in the comparison, the practical application of the same method and existing equipment of the present invention can not only provide facial recognition, but also provide further functions. Eye iris recognition, so the combination of two biometric features can provide a higher level of accurate identification.

The preferred implementation, as shown in Figures 4 and 5, is that the present invention uses a wide-area image signal 305 and at least two narrow-area image signals 301 and 302 to be superimposed and combined. To accurately calculate the depth of the measured object's three-dimensional features, gestures, obstacle avoidance, etc., which are very important for emerging 3D applications in the future, which can effectively provide current include: VR/AR, UAV, passenger flow calculation (People/Things Counting), etc. 3D depth image ranging function application, even with the ability to accurately detect objects and the surrounding environment depth measurement capabilities, so it can also provide future artificial intelligence (Artificial Intelligence), computer vision (Computer Vision) and other applications. For example, the recognition hardware mechanism 1 can be installed on a vehicle, and used for general automobile or motorcycle face recognition, door opening or fatigue detection, but the actual application is not limited to this.

In a preferred implementation, as shown in Figures 4 and 9, the image enhancement processing unit 312 of the present invention can be achieved by a software or firmware, which facilitates modification to increase the number of narrow-range images captured, or to adjust the light transmittance of the image , So that the light transmittance can be between 30% to 95%. For example, when capturing, one more narrow-area image signal 303 can be located in the wavelength range of 1050nm, so that by stacking three narrow-area image signals 301, 302, 303 with different spectral ranges separated by infrared light waves between 850nm, 940nm and 1050nm , Which makes the depth of the recognition level more obvious, and effectively enhances the three-dimensional sense and clarity of the entire image.

The above is only a convenient example. The number of narrow-area images added is not limited to this. In fact, it can be graded into a variety of specifications with different precisions, and customized to accurately identify whether the object 90 is biological or non-physical. The physical entity of a living thing. And it is widely used in various products such as security monitoring, industrial monitoring, face recognition equipment, network cameras, unmanned aerial vehicles, robots, and car-assisted image shooting.

In summary, the present invention can indeed achieve the aforementioned objectives, and it has actually complied with the provisions of the Patent Law, so a patent application was filed according to law. However, the above are only preferred embodiments of the present invention, and should not be used to limit the scope of implementation of the present invention; therefore, all simple equivalent changes and modifications made in accordance with the scope of the patent application of the present invention and the content of the patent specification, All should fall within the scope of the invention patent.

1‧‧‧Identify the hardware mechanism 100‧‧‧Identification System 2‧‧‧Multi-spectrum luminous unit 20‧‧‧Single multi-frequency LED 21‧‧‧Light Emitting Diode 3‧‧‧Multi-spectrum image sensing unit 30‧‧‧Single multi-frequency image sensor 31‧‧‧Image sensor 301‧‧‧Narrow field image signal 302‧‧‧Narrow field image signal 303‧‧‧Narrow field image signal 305‧‧‧Wide area video signal 310‧‧‧Sensitive pixel array 311‧‧‧Package circuit 312‧‧‧Image enhancement processing unit 4‧‧‧Single plane image 5‧‧‧3D relief image 6‧‧‧Preliminary Identification Learning Unit 60‧‧‧Original 65‧‧‧Reference three-dimensional relief image 7‧‧‧Ambient light enhanced comparison unit 70‧‧‧Ambient Light Sensor 90‧‧‧Object to be tested a1‧‧‧Infrared lens a2‧‧‧7 million pixel lens a3‧‧‧Flood Illuminator a4‧‧‧Proximity sensor a5‧‧‧Ambient Light Sensor a6‧‧‧dot projector

Figure 1 Schematic diagram of the identification system of the present invention. Figure 2 is a schematic diagram of a 3D relief image generated by the identification method of the present invention. Figure 3 A block diagram of the system of the present invention. Figure 4 Schematic diagram of a single multi-frequency image sensor of the present invention. Figure 5 The spectrum of the receiving range of a single multi-frequency image sensor of the present invention. Figure 6 The flow chart of the face recognition and comparison of the present invention. Figure 7 A schematic diagram of the present invention with an ambient light sensor. Figure 8 A diagram of an example of the application of the present invention to a smart phone. Figure 9 The spectrum of the receiving range of another embodiment of a single multi-frequency image sensor of the present invention. Figure 10 The configuration diagram of face recognition of the familiar Apple iPhoneX smart phone.

1‧‧‧Identify the hardware mechanism

100‧‧‧Identification System

2‧‧‧Multi-spectrum luminous unit

3‧‧‧Multi-spectrum image sensing unit

301‧‧‧Narrow field image signal

302‧‧‧Narrow field image signal

4‧‧‧Single plane image

90‧‧‧Object to be tested

Claims (11)

A method for multi-spectrum high-precision identification of an object mainly includes the following steps: building an identification hardware mechanism in an identification system, which has at least one multi-spectrum light-emitting unit and one multi-spectrum image sensing unit; and emits light with the multi-spectrum The unit emits light of different frequencies to project the object to be measured, the light emitted by the multi-spectrum light-emitting unit includes at least two infrared light, and the spectral range is distributed between 850nm and 1050nm; and the multi-spectrum image sensing unit The multi-spectral image sensing unit captures images projected by the object to be measured under different frequencies of light, and the multi-spectrum image sensing unit captures a narrow-area image signal including at least two infrared light reflections, and the spectral range of the multi-spectrum light-emitting unit corresponds to the interval distribution position Between 850nm and 1050nm, and the narrow-band image signal wavelength bandwidth of each infrared light is between 10nm and 60nm. The X-axis and Y-axis are a single plane image, and the different spectral wavelengths of the Z-axis are the image depths. The sampling wavelength of the Z axis includes at least two infrared light narrow-range image signals, the spectral range and the multi-spectral image sensing unit corresponding to the image at intervals between 850nm and 1050nm, and the frequency band of each infrared light is at least Between 10nm and 60nm; The multiple X-axis and Y-axis single plane images sampled in different bandwidth bands on the Z-axis are calculated and superimposed into a 3D relief image for accurate comparison and identification. For example, the method of multi-spectrum high-precision object identification described in the scope of patent application, wherein the image sensing unit is composed of a plurality of image sensors with different spectrums or a single multi-frequency image sensor, and the single multi-frequency image sensor The high-frequency image sensor mainly includes: a photosensitive image element array and its connected packaging circuit to drive and control the photosensitive image element array to capture external light and convert it into an output combined image signal, wherein the photosensitive image element array can capture all RGB Color visible light and IR infrared non-visible light for photoelectric conversion, and; an image enhancement processing unit built in the package circuit to control the image captured by the photosensitive pixel array, including: a full-color RGB visible light wide-area image Signal, the full-color RGB visible light wavelength spectrum range is between 400nm and 700nm, and at least two infrared non-visible light narrow-area image signals, the range interval distribution is between 850nm and 940nm, each of the infrared non-visible light The wavelength bandwidth of the narrow-area image signal is between 10 nm and 60 nm, and the wide-area image signal and the two narrow-area image signals are re-integrated and stacked to form a clear output image with front and rear hierarchical three-dimensional effects. For example, the multi-spectrum high-precision object identification method described in item 1 of the scope of patent application, wherein the multi-spectrum light-emitting unit is composed of a plurality of different frequency light-emitting diodes or a single multi-frequency light-emitting diode, and the single Multi-frequency light emitting diodes can emit at least two kinds of infrared light in the range of 850nm to 1050nm. For example, the method of multi-spectrum and high-precision identification of objects described in the scope of patent application, wherein the object to be tested is a biological or non-biological physical entity; and the identification method is further provided with a preliminary identification learning unit corresponding to the two Narrow infrared spectroscopy is set to 850nm and 940nm as the reference to take at least one image at the top, bottom, middle, left, and right angles of an original, and when the original moves in the cross position, at least one image is taken at each interval angle. The multiple X-axis and Y-axis single plane images sampled in the 850nm and 940nm wavelength bands of the narrow-band infrared spectroscopy with different bandwidths are calculated and combined into an original reference three-dimensional relief image for subsequent comparison and identification. For example, the method for multi-spectrum and high-precision object identification described in item 4 of the scope of patent application, wherein the preliminary identification learning unit further emits intermittent sounds or voices when executing, as the original object's corresponding upward, middle, downward, left and right angular displacement speed The reference instructions. For example, the method of multi-spectrum and high-precision object identification described in item 4 of the scope of patent application, wherein when the 3D relief image of the object to be measured is completed, it is first determined whether the object to be measured is a physical entity, if it is further stored with the preliminary identification learning unit The original object is compared with the three-dimensional relief image, and the comparison is correct to execute the activation. If it is not correct, the activation will not be performed. For example, the method for multi-spectrum and high-accuracy object identification described in item 4 of the scope of patent application, wherein the identification hardware mechanism is further provided with an ambient light sensor, and the identification method is further provided with an ambient light enhancement comparison unit, when the ambient light The sensor detects that the ambient light reaches the first dark level. At this time, the ambient light enhancement comparison unit is activated to compare the 3D relief image of the object to be measured with the original reference relief image obtained by infrared light at 940nm, and When the ambient light sensor detects that the ambient light reaches the second darker level, the ambient light enhancement and comparison unit automatically switches to obtain the original reference three-dimensional relief image from the 3D relief image of the object under test and the infrared light at 850nm. Comparing, adjust automatically according to different ambient light brightness, in order to obtain more accurate image recognition effect. For example, in the method for multi-spectrum high-precision object identification described in item 1 of the scope of patent application, the object to be measured is a human face. For example, the multi-spectrum high-precision object identification method described in item 1 of the scope of patent application, wherein the object to be tested is the human face and the iris of the eye. For example, the multi-spectrum high-precision object identification method described in item 1 of the scope of patent application, wherein the identification hardware mechanism is set on a smart mobile device. For example, the method of multi-spectrum high-precision object identification described in item 1 of the scope of patent application, wherein the identification hardware mechanism is set on a vehicle.

Images