WO2023151698A1

WO2023151698A1 - Handprint acquisition system

Info

Publication number: WO2023151698A1
Application number: PCT/CN2023/076009
Authority: WO
Inventors: 汤林鹏; 邰骋; 祝素伟; 陈军航; 张舒畅
Original assignee: 墨奇科技（北京）有限公司; 北京至简墨奇科技有限公司
Priority date: 2022-02-14
Filing date: 2023-02-14
Publication date: 2023-08-17

Abstract

Embodiments of the present application provide a handprint acquisition system. The handprint acquisition system comprises: a lighting system, which comprises one or more blue light sources and one or more green light sources, wherein each light source is used for emitting light to a handprint acquisition area, and the handprint acquisition area is used for placing a photographed object; a structured light projector, which is used for emitting structured light to the handprint acquisition area; one or more image acquisition devices, which are used for acquiring an image of the handprint acquisition area at the same time when the lighting system and the structured light projector emit light to the handprint acquisition area, so as to obtain a handprint image of the photographed object; a processing device, which is used for controlling the lighting system and the structured light projector to emit light and controlling the one or more image acquisition devices to collect the handprint image, and is further used for processing the handprint image acquired by the one or more image acquisition devices so as to obtain a structured light image and an illumination light image respectively corresponding to the image acquisition devices; and performing fusion and structured light image-based transformation on the illumination light image so as to obtain a processed handprint image.

Description

Handprint collection system

technical field

The present application relates to the technical field of non-contact handprint collection, and in particular, to a handprint collection system.

Background technique

Handprint recognition technology is one of many biometrics. The so-called biometric identification technology refers to the technology that uses the inherent physiological or behavioral characteristics of the human body to carry out personal identification. Due to the advantages of convenience and security of biometric identification, biometric identification technology has broad application prospects in the fields of identity authentication and network security. Available biometric identification technologies include handprint (such as fingerprints, etc.), face, voiceprint, iris, etc., among which handprint is the most widely used one.

Handprint recognition technology first needs to carry out the collection of handprint. In the existing handprint collection technology, the compatibility of different groups of people is not good enough. For example, the effect of collecting handprints under various conditions such as over-dry, over-wet and shallow handprints is limited, and it is difficult to meet the collection needs of various groups of people.

Contents of the invention

In order to at least partly solve the problems existing in the prior art, according to one aspect of the present application, a handprint collection system is provided, including: a lighting system, the lighting system includes one or more blue light sources and one or more green A light source, each light source is used to emit light toward the handprint collection area, and the handprint collection area is used to place the object to be photographed, and the object to be photographed includes at least one of single finger, double thumb, four fingers, flat palm, and side palm; a structured light projector, The structured light projector is used to emit structured light to the handprint collection area; one or more image acquisition devices, and one or more image acquisition devices are used to collect handprints while the lighting system and the structured light projector emit light to the handprint collection area The image of the collection area is obtained to obtain the handprint image of the subject; the processing device is used to control the lighting system and the structured light projector to emit light and control one or more image acquisition devices to collect the handprint image; the processing device is also used to process a Process the handprint images collected by one or more image acquisition devices to obtain the structured light images and illumination light images corresponding to one or more image acquisition devices; perform fusion processing on the illumination light images and transform processing based on the structured light images to obtain The processed handprint image; wherein, the illumination light image includes a blue light image and a green light image, and the image used for fusion processing includes at least one blue light image and at least one green light image, and the image corresponding to the image used for fusion processing The acquisition devices are the same, the shooting objects are the same, the shooting time interval is within the interval range, and the shooting lighting conditions are different, and the lighting conditions include at least one of the light source color and the lighting angle of the lighting system being different.

In the handprint collection system in the embodiment of the present application, the blue light source and the green light source are used to supplement light for the image collection device when collecting the handprint image. This can expand the handprint acquisition system to different skin The applicability of the skin condition, which is conducive to taking into account the collection needs of various groups of people.

Description of drawings

The following drawings of the application are hereby considered as part of the application for understanding the application. The accompanying drawings show the embodiments of the present application and their descriptions to explain the principles of the present application. In the attached picture,

Figure 1a shows a schematic diagram of a structured light projector according to an embodiment of the present application;

Figures 1b-1d show schematic diagrams of structured light patterns according to an embodiment of the present application;

2 shows a schematic diagram of a target acquisition system and related target objects and target acquisition areas according to an embodiment of the present application;

FIG. 3 shows a schematic diagram of a mosaic image obtained by directly superimposing and stitching multiple fingerprint images together in the prior art;

FIG. 4 shows a schematic flowchart of an image stitching method according to an embodiment of the present application;

Fig. 5 shows a schematic diagram of reconstructing a structured light pattern according to an embodiment of the present application;

FIG. 6 shows an example of a structured light pattern including a stitched structured light pattern according to an embodiment of the present application;

7a-7c show schematic diagrams of three structured light images corresponding to fingerprint acquisition at three different angles;

Fig. 8 shows a schematic diagram of a structured light image after nail removal according to an embodiment of the present application;

Fig. 9 shows a simple schematic diagram of collecting structured light stripes by a camera according to an embodiment of the present application;

Fig. 10 shows a schematic diagram of a visualized three-dimensional point cloud model according to an embodiment of the present application;

Figures 11a-11b show a schematic diagram of the deduplication operation before the model is expanded along the common coordinate axis according to an embodiment of the present application;

Fig. 12 shows a schematic diagram of a three-dimensional model unfolded along a common coordinate axis according to an embodiment of the present application;

Fig. 13 shows a schematic diagram of stitching two images;

Fig. 14 shows a schematic diagram of a handprint collection system and related fingers and handprint collection areas according to an embodiment of the present application;

Fig. 15 shows a schematic diagram of a part of the handprint collection system according to one embodiment of the present application;

Fig. 16 shows a schematic diagram of a partial structure of a handprint collection system and related fingers and handprint collection areas according to an embodiment of the present application;

Fig. 17 shows a schematic diagram of image acquisition and light source lighting sequence according to an embodiment of the present application;

Fig. 18 is a perspective view of a non-contact handprint collection device according to an exemplary embodiment of the present application;

Fig. 19 is a front view of a non-contact handprint collection device according to an exemplary embodiment of the present application;

Fig. 20 is a top view of a non-contact handprint collection device according to an exemplary embodiment of the present application;

Fig. 21 is a side view of a non-contact handprint collection device according to an exemplary embodiment of the present application;

Fig. 22 is an exploded view of a non-contact handprint collection device according to an exemplary embodiment of the present application;

Figure 23 is an axonometric view of the non-contact handprint collection device shown in Figure 22;

Fig. 24 is a cross-sectional view of the non-contact handprint collection device shown in Fig. 23;

Fig. 25 is an enlarged view of the handprint collection part of the non-contact handprint collection device shown in Fig. 22;

Fig. 26 is a top view of the handprint collection part shown in Fig. 25;

Fig. 27 is a front view of the handprint collection part shown in Fig. 25;

Fig. 28 is a side view of the handprint collection part shown in Fig. 25;

Fig. 29 is a rear view of the handprint collection part shown in Fig. 25;

Fig. 30 shows a schematic flowchart of a non-contact target object handprint collection method according to an embodiment of the present application;

Fig. 31A shows a schematic diagram of a structured light channel image according to an embodiment of the present application;

Fig. 31B shows a schematic diagram of the arrangement of structured light repeating units in structured light according to another embodiment of the present application;

Fig. 32A shows a schematic diagram of a structured light channel image according to another embodiment of the present application;

Fig. 32B shows a schematic diagram of arrangement of structured light repeating units in structured light according to another embodiment of the present application;

Fig. 33 shows a schematic diagram of processing objects in an unstructured light channel image according to an embodiment of the present application;

Figure 34 shows a schematic diagram of an unstructured light channel image according to one embodiment of the present application; and

Fig. 35 shows a schematic block diagram of a non-contact handprint collection device according to an embodiment of the present application.

Detailed ways

It will be appreciated by those skilled in the art that the following description is merely illustrative of preferred embodiments of the application and that the application may be practiced without one or more of these details. In addition, in order to avoid confusion with the present application, some technical features known in the art are not described in detail.

In order to at least partly solve the technical problem that the above-mentioned existing handprint collection technology is not good enough for different groups of people, an embodiment of the present application provides a handprint collection system. The handprint collection system includes an illumination system, a structured light projector, one or more image collection devices and a processing device.

The lighting system includes one or more blue light sources and one or more green light sources, each light source is used to emit light toward the handprint collection area, and the handprint collection area is used to place the object to be photographed, and the object to be photographed includes At least one of single finger, double thumb, four fingers, flat palm, and side palm.

The structured light projector is used for emitting structured light to the handprint collection area.

One or more image acquisition devices are used to collect images of the handprint collection area while the illumination system and the structured light projector emit light to the handprint collection area to obtain a handprint image of the subject.

The processing device is used to control the lighting system and the structured light projector to emit light and control one or more image acquisition devices to collect handprint images; the processing device is also used to process the handprint images collected by one or more image acquisition devices to obtain a Or the respective structured light image and illumination light image corresponding to a plurality of image acquisition devices; the illumination light image is fused and processed based on the transformation process of the structured light image to obtain the processed handprint image; wherein the illumination light image includes a blue light image ( That is, the following blue channel image) and the green light image (ie, the following green channel image), the images used for fusion processing include at least one blue light image and at least one green light image, and the images used for fusion processing correspond to The image acquisition devices are the same, the shooting objects are the same, the shooting (i.e. collecting) time interval is within the interval range, and the shooting lighting conditions are different. Different lighting conditions include at least one of the color of the light source and the lighting angle of the lighting system.

There may be multiple blue light sources and green light sources, and they may be arranged at different angles. In this way, by switching on and off the blue light source at different angles and/or adjusting the brightness, different blue light lighting conditions can be combined; similarly, different green light lighting conditions can be combined.

The interval range may be a preset time interval range, which may be set to an appropriate size as required. The interval range may be expressed as [t ₁ , t ₂ ], for example, t ₁ may be 0 second, and t ₂ may be 0.1 second, 0.2 second, 0.3 second or other time values close to 0. The shooting time of the images used for fusion processing may be the same or very close, that is, the upper limit of the interval range may be set to a time value close to 0.

The processing device can perform the following operations: 1. It is used to control the lighting system and the structured light projector to emit light and control one or more image acquisition devices to collect handprint images. That is, the processing means can be used to control the capture. The way to control shooting may include pre-setting multiple different lighting conditions for shooting, for example, shooting P1 under the first lighting condition at t1, shooting P2 under the second lighting condition at t2, shooting P3 under the second lighting condition at t3, and then The better quality parts of P1, P2, and P3 can be merged and shot. The method of controlling shooting may also include shooting first, and then shooting after adjusting the lighting conditions according to the captured image. For example, when P1 is photographed under the first lighting condition at t1, and it is determined through image analysis that the lower left corner of P1 is darker, the corresponding light intensity at the lower left corner of P1 may be increased. The method of controlling the shooting may also include determining whether the posture of the shooting object is qualified through the preview image acquisition device, and then starting the shooting after passing it. For example, the structured light image can be taken under the condition of only using the structured light source, and the attitude of the subject can be judged by the structured light image, or the image can be taken under the condition of using the structured light source and the illumination light source, and the structured light image and the illumination The light image captures the pose of the subject. 2. It is used to process the handprint images collected by one or more image acquisition devices, and obtain the corresponding structured light images and illumination light images of one or more image acquisition devices; for example, the images obtained by shooting are RGB three-channel images , wherein the image of the red channel is a structured light image, and the images of the blue and green channels are illumination light images; 3. Perform subsequent processing such as fusion processing and transformation processing based on the illumination light image and the structured light image. For subsequent processing, the collected All images are used for subsequent processing, or a part of images may be selected for subsequent processing based on image quality, subject pose, and target image acquisition device (for example, a highly suitable image acquisition device).

The fusion processing of the illumination light images may include fusion of images under blue light source and green light source, and optionally may also include fusion of images under different illumination angles of the same color light source. Images under the same light source with the same light angle and different light intensities may not be fused.

For example, for the blue-light image B1 taken by camera C1 at time t1, and the blue-light image B2 taken by camera C1 at time t2 (the illumination angles of B1 and B2 are different, if B1 and B2 are only different in light intensity, fusion is not required), camera C1 The green light image G1 taken at time t1, and the green light image G2 taken by camera C1 at time t2 (G1 and G2 have different illumination angles), several fusion scenarios include but are not limited to:

In case 1, the images used for fusion can be B1 and G1, and BG1 is obtained through fusion. At this time, the camera corresponding to the image used for fusion is camera C1.

Case 2, the images used for fusion can be B1, G1, B2, G2: you can first fuse B1+G1=BG1, B2+G2=BG2, and then fuse BG1 and BG2 (BG1 and BG2 are all blue-green fusion image, which belongs to the fusion of the same color and different lighting angles). When fusing B1, G1, B2, and G2, it is preferable to fuse B1 and G1, B2 and G2 that were shot at the same time first, instead of fusing B1 and B2 first, because B1 and B2 are shot at different times, and the shooting object is between t1-t2 There will be a certain displacement. At this time, the camera corresponding to the image used for fusion is camera C1.

In case 3, the images used for fusion may be B1' obtained by transforming B1 and G1' obtained by reconstructing G1: B1'+G1'=BG1'. At this time, the camera corresponding to the image used for fusion is camera C1.

Situation 4. It is assumed that camera C2 also photographed B1 and G1 at time t1, and photographed B2 and G2 at time t2. At this time, the image used for fusion can be: B1' obtained after B1 transformation taken by camera C1 and B1' obtained after B1 transformation taken by camera C2 and B1P obtained after splicing; G1' obtained after G1 transformation taken by camera C1 G1P obtained after splicing with G1' obtained after transformation of G1 captured by camera C2. It will be possible to fuse B1P and G1P. At this time, the cameras corresponding to B1P used for fusion are camera C1 and camera C2, the cameras corresponding to G1P are camera C1 and camera C2, and the cameras corresponding to B1P and G1P are the same.

Exemplarily, the wavelength of the blue light emitted by the blue light source is less than 430 nm, and the wavelength of the green light emitted by the green light source is greater than 540 nm.

Exemplarily, performing fusion processing on the illumination light image and transformation processing based on the structured light image may include: performing fusion processing on the blue light image and the green light image in at least one illumination light image captured by the same image acquisition device to obtain a fusion image ; Based on the structured light image corresponding to at least one illumination light image, transforming the fused image to obtain a processed handprint image.

This solution is to first fuse the blue-light image and the green-light image taken by the same image acquisition device, and then transform the fused image. The transformation process may be a reconstruction transformation based on reconstructed structured light patterns such as described below.

Exemplarily, performing the fusion processing on the illumination light image and the transformation processing based on the structured light image may include: based on the structured light image corresponding to at least one illumination light image captured by the same image acquisition device, for the at least one illumination light image The blue image is transformed to obtain the transformed blue image, based on the structured light image corresponding to at least one illumination light image taken by the same image acquisition device, the green image in the at least one illumination light image is transformed to obtain the transformed The transformed green image is merged with the transformed blue image and the transformed green image to obtain the processed handprint image.

In this solution, the blue-ray image and the green-light image are transformed first, and then the transformed images are fused.

Exemplarily, at least one illumination light image includes a first illumination light image captured at a first moment and a second illumination image captured at a second moment; the blue light image and the green light image in at least one illumination light image captured by the same image acquisition device Performing fusion processing on the images to obtain a fusion image, including: performing fusion processing on the first blue-light image and the first green-light image in the first illumination light image captured by the same image acquisition device at the first moment, to obtain the first blue-green fusion image , performing fusion processing on the second blue-light image and the second green-light image in the second illumination light image captured by the same image acquisition device at the second moment to obtain a second blue-green fusion image, for the first blue-green fusion image and the second The two blue-green fused images are fused to obtain a fused image.

This solution is to fuse the four images of B1, G1, B2, and G2 (first B1+G1, B2+G2, and then BG1+BG2), and then transform the fused images.

Exemplarily, at least one illumination light image includes a first illumination light image taken at a first moment and a second illumination image taken at a second moment; the transformed blue image includes the first blue color in the first illumination light image A first transformed blue image determined from the image and a second transformed blue image determined from the second blue image in the second illuminated light image, the transformed green image comprising the first green image determined from the first illuminated light image The determined first transformed green image and the second transformed green image determined by the second green image in the second illumination light image are fused with the transformed blue image and transformed green image to obtain the processed hand The texture image includes: performing fusion processing on the first transformed blue image and the first transformed green image to obtain the first transformed fusion image, and performing fusion processing on the second transformed blue image and the second transformed green image to obtain the second transformed image The images are fused, performing fusion processing on the first transformation fusion image and the second transformation fusion image to obtain a processed handprint image.

This scheme is to obtain B1', B2', G1', G2' through transformation processing, and then fuse 4 images (first B1'+G1' to get BG1', B2'+G2 to get BG2', and then BG1'+BG2 ').

Exemplarily, the illumination light image includes a first illumination light image taken by the same image acquisition device at the first moment and a second illumination image taken at the second moment; fusion processing and transformation processing based on the structured light image are performed on the illumination light image, Obtaining the processed handprint image includes: performing fusion processing on the first blue image in the first illumination light image and the first green image in the first illumination light image to obtain the first blue-green fusion image; performing fusion processing on the second blue image in the illumination light image and the second green image in the second illumination light image to obtain a second blue-green fusion image; Based on the first structured light image corresponding to the first illumination light image, the first blue-green fusion image is transformed to obtain the first fusion transformation image; based on the second structured light image corresponding to the second illumination light image, the second blue-green fusion image is obtained. performing transformation processing on the green fusion image to obtain a second fusion transformation image; performing fusion processing on the first fusion transformation image and the second fusion transformation image to obtain a processed handprint image.

This scheme is: B1+G1=BG1, then BG1 is transformed so that BG1->(BG1)'; B2+G2=BG2, then BG2 is transformed so that BG2->(BG2)'; then Fusion (BG1)'+(BG2)': (BG1)'+(BG2)'.

Exemplarily, the fusion processing includes calculation fusion and/or neural network model fusion; calculation fusion includes: calculating the sharpness index of the pixel at the first position in the image used for fusion processing, and using the sharpness index in the image for fusion processing The pixel value of the pixel at the first position with a better index is used as the pixel value of the pixel at the first position in the obtained image after fusion; the neural network model fusion includes: inputting the pixel at the second position in the image for fusion processing into the neural network The network model obtains the pixel value of the pixel at the second position in the fused image.

For example, the sharpness index of the pixel at the first position in the blue image and the green image are respectively calculated, if the sharpness index of the pixel at the first position in the blue image is better than the sharpness index of the pixel at the first position in the green image, Then the pixel value of the pixel at the first position in the fused image is the pixel value of the pixel at the first position in the blue image.

The calculation method of the sharpness index at the first position is, for example, calculating the standard deviation between the pixel value of the pixel at the first position and the pixel value of the pixels in the neighborhood around the first position, and using the standard deviation as the sharpness index corresponding to the pixel at the first position. The larger the standard deviation, the more drastic the change of the pixel value in the neighborhood around the first position, and the higher the definition. Alternatively, the standard deviation between the pixel values of the pixels in the first position is used as the sharpness index corresponding to the pixel in the first position. Exemplarily, the sharpness index may also be represented by the following quality score.

The pixel at the first position may be a single pixel or a plurality of pixels in an image area (for example, a 3*3 area), so as to perform image fusion at the pixel level/image area level. When the pixel at the first position is a single pixel, each pixel position in the image can be taken as the first position in turn; when the pixel at the first position is multiple pixels in one image area, the image can be divided into multiple areas, and the Use each region as the first location.

Exemplarily, all the positions in the image used for fusion processing may be used as the second positions or some positions that need to be fused are used as the second positions (for example, the positions that do not need to be fused are masked) to perform the above neural network model fusion. For example, B1, G1, B2, and G2 that need to be fused are input into the neural network to obtain a fused image.

Optionally, after fusion, pixel value optimization may be performed on boundary regions in the image obtained after fusion from different images used for fusion processing. Through pixel value optimization, the connection can be made smoother and continuous. For example, the first position is a 3*3 area, the entire image is 9*9, and the entire image contains 1-9 9 first positions, for example, the first first position in the fusion image comes from the blue image, and the first position in the fusion image The 2 first positions are from the green image, then rows 1-3 column 3 and The pixels in the 1st-3rd row and the 4th column are the pixels in the border area, and the pixel value optimization can be performed on the pixels in the border area.

Computational fusion and neural network model fusion can be performed arbitrarily or combined. Exemplarily, the fusion process includes calculation fusion and neural network model fusion, and the calculation fusion includes: calculating the sharpness index of the pixel at the first position in the image used for fusion processing, if the multiple images used for fusion processing are clear If the gap of the sharpness index is greater than the gap threshold, the pixel value of the pixel at the first position with a better sharpness index in the image for fusion processing is used as the pixel value of the pixel at the first position in the obtained image after fusion; the neural network model The fusion includes: inputting the pixel at the second position in the image for fusion processing into the neural network model to obtain the pixel value of the pixel at the second position in the image obtained after fusion; the second position is a plurality of images used for fusion processing The position where the gap of the medium sharpness index is not greater than the gap threshold.

Computational fusion and neural network model fusion can be used in combination. For example, for the pixels at the same position in the blue-light image and the green-light image, if the difference between the sharpness index of the pixel in the blue-light image and the sharpness index of the pixel in the green-light image is greater than the gap threshold, it means that the pixel at this position in the blue-light image is If the sharpness is significantly better than that of the pixel at this position in the green light image, then the pixel value of the pixel at this position in the blue light image is used as the pixel value of the pixel at this position in the fusion image; on the contrary, if the pixel value of the pixel at this position in the green light image If the difference between the sharpness index and the sharpness index of the pixel in the blue-ray image is greater than the gap threshold, it means that the sharpness of the pixel at this position in the green-light image is significantly better than that of the pixel at this position in the blue-light image. The pixel value of the pixel at the position is used as the pixel value of the pixel at the position in the fused image. If the difference between the two is smaller than the threshold, it means that the sharpness indicators of the two are similar. At this time, the neural network model fusion method can be used to determine the pixel value of the fused image at the position. Exemplarily, still taking the 9*9 fused image as an example, the fused image includes 81 pixels. For pixels at positions 1-40, the clarity index of the blue image is significantly better than that of the green image, and for pixels at positions 51-81, the clarity index of the green image is significantly better than that of the blue image, and at positions 41-50 pixel, and the blue image definition index is similar to the green image definition index, then cover the blue image except for positions 41-50 (for example, set it to 0), and the green image except for positions 41-50 The position of is masked (for example, set to 0) and input to the neural network model fusion to obtain the 41st-50th pixels in the fusion image. The pixels at positions 1-40 in the fused image are from the blue image, and the pixels at positions 51-81 are from the green image. It is understandable that, after the combination of computational fusion and neural network model fusion, the pixel values from the boundary area in the fused image can also be optimized.

The neural network model used for image fusion can adopt existing model architecture and training methods. For example, the fused image obtained by the calculation method is used as the real value, the difference between the predicted fused image of the neural network model and the real value is used to determine the loss value, and the neural network model is updated with the loss value. Another example is to perform expert scoring on the fused images predicted by the neural network model, so that the neural network model can adjust network parameters to obtain better scores.

The above-mentioned one or more image acquisition devices have at least the following three setting schemes:

A1: Three image capture devices can be set up to capture images of a single finger. For example, you can The fingerprint images of a single finger are respectively collected by three image acquisition devices whose optical axes are at a certain angle to each other, and the fingerprint images respectively collected by the three image acquisition devices can be spliced to simulate rolling fingerprints.

A2: Six image acquisition devices can be provided, including three image acquisition devices whose optical axes are at an angle to each other and three image acquisition devices whose optical axes are substantially parallel, and optionally a preview image acquisition device. The above six cameras and the optional preview image collection device can be used to collect single finger, double thumb, four finger, and palm prints.

In this case, when collecting a single finger, the fingerprint images collected by the image acquisition device with three optical axes at an angle to each other can be spliced to simulate the rolling fingerprint of a single finger; Among the image acquisition devices with substantially parallel axes, select an image acquisition device with a suitable position for collecting fingerprint images; when collecting palmprints, three image acquisition devices with substantially parallel optical axes can be used to collect palmprint images respectively, and the collected palmprints The images are stitched together to obtain a complete palm print.

A3: Multiple (for example, 2) image capture devices with partially overlapping depth-of-field ranges can be provided, and one image capture device for preview can optionally be provided. The above-mentioned image capture means with partially overlapping depth-of-field ranges and optional preview image capture means can be used to capture two thumbs and four fingers. When capturing double thumbs and four fingers, images captured by an image capturing device with a suitable height can be selected from multiple image capturing devices with partially overlapping depth-of-field ranges for processing.

Exemplarily, one or more image acquisition devices include a plurality of common image acquisition devices that jointly photograph the same subject, perform fusion processing on the illumination light image and transformation processing based on the structured light image, and obtain the processed handprint image , including: performing fusion processing on the illumination light image, transformation processing based on the structured light image, and splicing the transformed images corresponding to the common image acquisition device to obtain the processed handprint image; the shooting objects include single finger, flat Palm or side palm.

Images collected by multiple common image collection devices can be spliced together to obtain a complete handprint. Stitching can be achieved using various image stitching schemes described below. Exemplarily, in an application scenario where it is necessary to obtain a single-finger rolling fingerprint or obtain a palm print, image stitching can be performed.

The fusion, transformation processing and splicing of the images can be performed in various orders, and the splicing must be performed on the transformed images. The execution order of the fusion, transformation and splicing steps is exemplified below.

Exemplarily, performing fusion processing on the illumination light image, transformation processing based on the structured light image, and splicing the transformed images corresponding to the common image acquisition device to obtain the processed handprint image may include: For multiple Each image acquisition device in the common image acquisition device performs fusion processing on the blue image and the green image in at least one illumination light image corresponding to the image acquisition device to obtain a fusion image corresponding to the image acquisition device; based on the image acquisition The structured light image corresponding to the device is transformed into the fusion image corresponding to the image acquisition device to obtain the transformation fusion image corresponding to the image acquisition device; the transformation fusion images corresponding to multiple common image acquisition devices are spliced to obtain the processed Post-print image.

This scheme belongs to the scheme of blue-green fusion first, then transformation, and then splicing.

Still taking B1 and G1 captured by the image acquisition device C1 and B1 and G1 captured by the image acquisition device C2 as an example, wherein C1 and C2 are common image acquisition devices. First fuse B1 and G1 shot by C1 to get BG1 shot by C1, fuse B1 and G1 shot by C2 to get BG1 shot by C2, transform BG1 and BG2 to get BG1' shot by C1 and BG1 shot by C2 ', splicing BG1' taken by C1 and BG1' taken by C2 to obtain the processed handprint image.

Exemplarily, performing fusion processing on the illumination light image, transformation processing based on the structured light image, and splicing the transformed images corresponding to the common image acquisition device to obtain the processed handprint image may include: For multiple Each image acquisition device in the common image acquisition device, based on the structured light image corresponding to the image acquisition device, performs conversion processing on the blue image in at least one illumination light image corresponding to the image acquisition device to obtain the image acquisition device For the corresponding converted blue image, based on the structured light image corresponding to the image acquisition device, the green image in at least one illumination light image corresponding to the image acquisition device is converted to obtain the converted green image corresponding to the image acquisition device, Perform blue-green fusion processing on the converted blue image corresponding to the image acquisition device and the converted green image corresponding to the image acquisition device to obtain a transformed fusion image corresponding to the image acquisition device; transform and fuse images corresponding to multiple common image acquisition devices Splicing is performed to obtain the processed handprint image.

This scheme belongs to the scheme of transforming first, then blending blue and green, and then splicing.

Still taking B1 and G1 captured by the image acquisition device C1 and B1 and G1 captured by the image acquisition device C2 as an example, wherein C1 and C2 are common image acquisition devices. First, transform B1 and G1 taken by C1, B1 and G1 taken by C2 to obtain B1' and G1' taken by C1, B1' and G1' taken by C2, and then perform blueprinting on B1' and G1' taken by C1. Green fusion to obtain BG1' taken by C1, and BG1' taken by C2 are also obtained, and BG1' taken by C1 and BG1' taken by C2 are spliced to obtain a processed handprint image.

Exemplarily, performing fusion processing on the illumination light image, transformation processing based on the structured light image, and splicing the transformed images corresponding to the common image acquisition device to obtain the processed handprint image may include: For multiple Each image acquisition device in the common image acquisition device, based on the structured light image corresponding to the image acquisition device, performs conversion processing on the blue image in at least one illumination light image corresponding to the image acquisition device to obtain the image acquisition device For the corresponding converted blue image, based on the structured light image corresponding to the image acquisition device, the green image in at least one illumination light image corresponding to the image acquisition device is converted to obtain the converted green image corresponding to the image acquisition device; Splicing the converted blue images corresponding to a plurality of common image acquisition devices to obtain a spliced and transformed blue image; splicing the transformed green images corresponding to a plurality of image acquisition devices to obtain a spliced and transformed green image; splicing and transforming the blue images and The green image is spliced and transformed for fusion processing, and the processed handprint image is obtained.

This scheme belongs to the scheme of transforming first, then splicing, and then blending blue and green. Compared with the previous two schemes, this scheme is preferable because the number of images to be fused is relatively small and the used The image resolution is also lower (the original image resolution is higher), which can effectively reduce the calculation amount of blue-green fusion.

Still taking B1 and G1 captured by the image acquisition device C1 and B1 and G1 captured by the image acquisition device C2 as an example, wherein C1 and C2 are common image acquisition devices. First, transform B1, G1 taken by C1, B1 and G1 taken by C2 to obtain B1' and G1' taken by C1, B1' and G1' taken by C2, and then B1' taken by C1 and B1 taken by C2 'Stitching to obtain B1'P, splicing G1 taken by C1 and G1' taken by C2 to obtain G1'P, performing blue-green fusion on B1'P and G1'P to obtain the processed handprint image.

Exemplarily, the structured light projector includes a light source, a pattern generating unit and a converging lens, the pattern generating unit is arranged in front of the light source, and the light source is used to project the pattern on the pattern generating unit onto the projection plane to form a reconstruction structure on the projection plane The light pattern and the spliced structured light pattern, the converging lens is arranged on the light transmission path between the pattern generating unit and the projection plane; the reconstructed structured light pattern is different from the spliced structured light pattern, the reconstructed structured light unit and the spliced structure in the reconstructed structured light pattern There is no border overlap between the spliced structured light units in the light pattern; the reconstructed structured light unit is used to perform transformation processing based on the structured light image, and the spliced structured light unit is used to transform and process the image corresponding to multiple common image acquisition devices to splice.

The reconstruction transformation and stitching can both use the reconstruction structured light unit, or use different structured light units for reconstruction transformation and stitching.

Exemplarily, the reconstructed structured light unit and the stitched structured light unit meet at least one of the following conditions: the reconstructed structured light unit includes stripes; the stitched structured light unit includes scattered points; the distribution density of the stitched structured light unit is greater than the distribution of the reconstructed structured light unit density.

In this way, reconstructed structured light units with a smaller distribution density can be used for reconstruction and transformation to improve the reconstruction speed, and stitched structured light units with a larger distribution density can be used for fine splicing locally to improve splicing accuracy.

Exemplarily, a plurality of common image capture devices include a first common image capture device, a second common image capture device and a third common image capture device, the optical axis of the first common image capture device and the orientation of the handprint capture area are multiple The plane of the common image acquisition device is vertical, the optical axis of the second common image acquisition device forms a first preset angle with the optical axis of the first common image acquisition device, and the optical axis of the third common image acquisition device and the first common image acquisition The optical axis of the device forms a second preset included angle; the shooting object includes a single finger.

This scheme may correspond to the setup scheme A1 of the image acquisition device described above. The first common image capture device, the second common image capture device and the third common image capture device can be applied to a scene where it is necessary to obtain a single-finger rolling fingerprint. The first common image capture device, the second common image capture device and the third common image capture device may be respectively the first image capture device, the second image capture device and the third image capture device described below. The first common image capture device, the second common image capture device and the third common image capture device, these three may be a plurality of fifth image capture devices 1830 described below or include a plurality of fifth image capture devices described below In 1830.

Exemplarily, the plurality of common image capture devices include a fourth common image capture device, a fifth common image capture device and a sixth common image capture device; the fourth common image capture device, the fifth common image capture device and the sixth common image capture device The lens of the collection device is located in a predetermined plane and is respectively aimed at a plurality of first sub-regions in the handprint collection area, and any adjacent two of the plurality of first sub-regions overlap or adjoin each other; the shooting objects include flat palms or side palm.

This scheme may correspond to the arrangement scheme A2 of the above-mentioned image acquisition device. The fourth common image acquisition device, the fifth common image acquisition device and the sixth common image acquisition device can be applied to scenarios where palmprints need to be acquired. When the handprint collection device is only equipped with the fourth common image collection device, the fifth common image collection device and the sixth common image collection device, the handprint collection device can collect palmprints, but cannot be used to collect single-finger simulated rolling fingerprints. When the first common image acquisition device, the second common image acquisition device and the third common image acquisition device are further arranged, it is possible to collect single-finger rolling fingerprints and collect palm prints (and fingerprints).

The optical axes of the fourth common image capture device, the fifth common image capture device and the sixth common image capture device may be arranged parallel to each other, and the three may be located in the same predetermined plane. The image acquisition ranges (field of view) of two adjacent image acquisition devices among the fourth common image acquisition device, the fifth common image acquisition device and the sixth common image acquisition device may partially overlap or adjoin. Each image acquisition device in the fourth common image acquisition device, the fifth common image acquisition device and the sixth common image acquisition device can acquire images of at least part of the area in the handprint acquisition area, so that the fourth common image acquisition device, the fifth common image acquisition device The common image acquisition device and the sixth common image acquisition device can cooperate with each other to acquire images in the entire handprint acquisition area. The fourth common image capture device, the fifth common image capture device and the sixth common image capture device, these three can be a plurality of fourth image capture devices 1820 described below or include multiple fourth image capture devices described below In 1820.

Exemplarily, the one or more image acquisition devices include a plurality of independent image acquisition devices that each shoot the same object, the clearly imageable object surface subspaces of the plurality of independent image acquisition devices partially overlap, and the processing device is also used to The structured light image and/or illumination light image collected by the image acquisition device is determined from a plurality of independent image acquisition devices, and the target image acquisition device that can clearly image the subspace of the object surface and match the position of the photographed object is determined; the image used for fusion processing is obtained by Captured by the target image capture device; the shooting object includes two thumbs or four fingers; the preview image capture device is all image capture devices among multiple independent image capture devices, or the image capture device with the largest field of view among multiple independent image capture devices, or An image capture device that is different from multiple independent image capture devices and has a shorter focal length than the multiple independent image capture devices.

Multiple (for example, 3) common image acquisition devices can respectively capture a part of fingers or palms, and the images captured by multiple common image acquisition devices can be spliced to obtain a complete handprint.

The application scenario of multiple independent image capture devices may be the capture scenario for two thumbs or four fingers. It is understandable that the scene of collecting two thumbs or four fingers means that the shooting object is two thumbs or four fingers, and the images of two thumbs or four fingers need to be collected at the same time, and the processed handprint image can be Two single-finger images for two thumbs or four single-finger images for four fingers. When taking pictures of two thumbs or four fingers, the position, height and angle of each finger can be placed in the handprint collection area more flexibly. , height, and angle will have many restrictions, which will reduce the user experience. Therefore, it is hoped that a greater depth of field/field of view can be obtained through the combination of multiple image acquisition devices. Where the user places the finger, the image acquisition device corresponding to the position is selected for acquisition, which can reduce the need for the user's hand placement, height, Angle restrictions. , using the two thumbs or four fingers as a whole, or each of the two thumbs and four fingers as the camera selection unit, for each camera selection unit, select from a plurality of independent image acquisition devices to match the height/position of the camera selection unit The images collected by an image acquisition device are processed subsequently (without the need for multi-camera stitching) to obtain the final handprint image. For example, using a single finger as a camera selection unit, when four fingers are used together corresponding to multiple independent image acquisition devices, a larger depth of field/field of view can be obtained. When the clearly imageable object plane subspaces overlap in a direction parallel to the optical axis of the image acquisition device (for example, longitudinally), a greater depth of field can be obtained through multiple independent image acquisition devices. When the clearly imageable object surface subspaces overlap in a direction perpendicular to the optical axis of the image acquisition device (for example, laterally), a larger field of view can be obtained through multiple independent image acquisition devices. When the clearly imageable object surface subspace overlaps in a direction (such as longitudinal direction) parallel to the optical axis of the image acquisition device, the "object position ” can be the height of the subject. When the clearly imageable object surface subspace overlaps in the direction (such as lateral direction) perpendicular to the optical axis of the image acquisition device, the "object position ” can be the horizontal position of the subject.

It can be understood that multiple independent image acquisition devices can be used for image acquisition at the same time (in this case, the multiple independent image acquisition devices are all preview image acquisition devices), and the target image acquisition device corresponding to each camera selection unit is determined according to the acquired images , use the image captured by the target image capture device for subsequent processing; it is also possible to use one of multiple independent image capture devices as a preview image capture device for image capture, and determine the target image capture corresponding to each camera selection unit according to the captured image device, use the image captured by the target image capture device (if the target capture device is not a preview image capture device, then use the target capture device to re-acquire) for subsequent processing; it can also be performed with a preview image capture device other than multiple independent image capture devices Image acquisition: determine the target image acquisition device corresponding to each camera selection unit according to the acquired image, use the target image acquisition device to perform image acquisition, and use the image acquired by the target image acquisition device to perform subsequent processing.

In the case of using only a plurality of independent image acquisition devices, only the fingerprints of two thumbs and four fingers can be collected, which corresponds to the application scenarios of thumb and four fingers in the configuration schemes A3 and A2 of the image acquisition devices.

In the case of adopting a plurality of common image acquisition devices and a plurality of independent image acquisition devices, the fingerprints of double thumbs and four fingers can be collected, and the fingerprints and palm prints of single finger rolling can also be collected. When collecting the fingerprints of four fingers, the common image acquisition device adopted when collecting palm prints can be used as a collection Four-finger independent image acquisition device to use.

Exemplarily, the plurality of image acquisition devices includes a plurality of first independent image acquisition devices; the lenses of the plurality of first independent image acquisition devices are arranged around the center line and face the handprint acquisition area, wherein the plurality of seventh first independent images The acquisition devices respectively have clear imageable object surface subspaces within the range of depth of field from the front and back of the best object, and the clear imageable object surface subspaces of multiple first independent image acquisition devices partially overlap, and the multiple first independent image acquisition devices The clear imageable object surface subspaces together form the clear imageable total space, the handprint collection area includes the clear imageable object surface total space, and the best object planes of multiple first independent image acquisition devices have different positions in the handprint collection area ; the focal lengths of the lenses of the multiple first independent image capture devices are different; and/or, the distances from the front ends of the lenses of the multiple first independent image capture devices to the handprint collection area are different.

The first independent image acquisition device may be the seventh image acquisition device 2220 described below, that is, the plurality of first independent image acquisition devices may be the plurality of seventh image acquisition devices 2220 described below.

When the preview image acquisition device is the image acquisition device with the largest field of view among the plurality of independent image acquisition devices, it may be the image acquisition device with the shortest focal length among the plurality of independent image acquisition devices. If the focal lengths of the plurality of independent image acquisition devices are the same, the preview image acquisition device may be any image acquisition device or all image acquisition devices in the plurality of independent image acquisition devices.

According to the structured light image and/or illumination light image collected by the preview image acquisition device, the target image acquisition device that can clearly image the subspace of the object surface and match the position of the photographed object is determined from multiple image acquisition devices, including:

Determine the height of the subject according to the structured light image collected by the preview image acquisition device; determine the target image acquisition device that can clearly image the subspace of the object surface and match the height of the subject according to the height of the subject; or, according to the structured light collected by the preview image acquisition device The image determines the height of each finger included in the photographed object; according to the height of each finger, a target image acquisition device that can clearly image the subspace of the object surface and matches the height of each finger is determined.

When determining the height of the object to be photographed, the height of the entire four fingers can be determined (the camera selection unit is a whole of four fingers at this time), and a suitable image acquisition device is determined as the target image acquisition device according to the height of the four fingers. When determining the height of the object to be photographed, the height of each finger can also be determined (the camera selection unit is a single hand at this time), and a different image acquisition device is determined for each finger as the target image acquisition device.

Exemplarily, the plurality of image acquisition devices include a plurality of eighth image acquisition devices, the lenses of the plurality of eighth image acquisition devices are located in a predetermined plane and are respectively aimed at the plurality of first sub-regions in the handprint acquisition area, and the plurality of Any adjacent two of the first subregions overlap or adjoin each other.

When the preview image acquisition device is the image acquisition device with the largest field of view among the multiple independent image acquisition devices, if the multiple independent image acquisition devices have the same field of view, the preview image acquisition device can be any one of the multiple independent image acquisition devices or All image capture devices.

According to the structured light image and/or illumination light image collected by the preview image acquisition device, determine the target map that can clearly image the subspace of the object surface and match the position of the shooting object from multiple image acquisition devices Like acquisition devices, including:

Determine the horizontal position of the subject (such as the whole of the two thumbs or the whole of the four fingers) according to the image collected by the preview image acquisition device; determine the target image acquisition device that can clearly image the subspace of the object surface and match the horizontal position of the subject according to the horizontal position of the subject;

Or, determine the horizontal position of each finger contained in the subject according to the image collected by the preview image acquisition device; determine the target image acquisition device that can clearly image the subspace of the object surface and match the horizontal position of each finger according to the horizontal position of each finger.

For example, according to determining the position, posture and integrity of each finger in the image from the images taken by each independent image acquisition device, for example, the little finger is only captured by one image acquisition device, then this image acquisition device is used as the target image acquisition device for the little finger If the middle finger is captured by two image acquisition devices, then select the image acquisition device whose middle finger position is more centered in the images taken by the two image acquisition devices, the integrity of the middle finger captured or the middle finger posture is better as the target image acquisition device .

When determining the horizontal position of the shooting object, the horizontal position of the entire four fingers can be determined, and a suitable image acquisition device is determined as the target image acquisition device according to the horizontal position of the four fingers. When determining the horizontal position of the shooting object, the horizontal position of each finger may also be determined, and a different image acquisition device is determined for each finger as the target image acquisition device. In the field of image processing, after collecting images for different parts of a three-dimensional target, in order to obtain a complete target image, it is generally necessary to align and stitch multiple acquired images from different angles. In the prior art, there is a lack of an alignment reference between two images during alignment and stitching, and usually the two images are directly superimposed and stitched together. This splicing effect is not good, which affects the accuracy of subsequent target recognition (such as face recognition or fingerprint recognition).

In order to at least partly solve the above technical problems, embodiments of the present application provide a structured light projector and an image acquisition system. The structured light emitted by the structured light projector can form a variety of different structured light patterns. Using the image acquisition system of the structured light projector, different structured light patterns can be projected on the surface of the target object (that is, the three-dimensional target described in this paper), and the two-dimensional target image of the target object can be obtained based on different structured light patterns. The conversion relationship between the data to be spliced (such as a 2D expanded image or a 3D target model) and the precise alignment relationship between the data to be spliced, and based on the precise alignment relationship, the splicing between the data to be spliced is automatically obtained. Data (for example, an overall expanded image after stitching multiple 2D expanded images or an overall target model after stitching multiple 3D target models). The system can obtain high-precision overall data without much human intervention.

Although this paper mainly uses fingerprints as an example to describe the image acquisition system according to the embodiment of the application, it can be understood that the image acquisition system according to the embodiment of the application is not limited thereto, and it can be applied to any type of target Image collection, the target includes but not limited to fingerprints, palm prints, faces, etc. In one or some embodiments, the image collection system may be a handprint collection system for collecting fingerprints and/or palmprints and the like. The "handprint collection system" described herein may also be referred to as "handprint collection device/equipment" or "non-contact handprint collection device/equipment/system", etc.

According to one aspect of the present application, a structured light projector is provided. The structured light projector includes A light source, a pattern generating unit and a converging lens, the pattern generating unit is arranged in front of the light source, the light source is used to project the pattern on the pattern generating unit onto the projection plane, and the converging lens is arranged on the light transmission path between the pattern generating unit and the projection plane Above; the structured light beam emitted by the structured light projector forms a reconstructed structured light pattern and a spliced structured light pattern on the projection plane. The reconstructed structured light pattern is different from the spliced structured light pattern. The reconstructed structured light unit and the spliced structured light There is no border overlap between the tiled structured light units in the pattern.

A detailed description is given below with reference to FIGS. 1a-1d. Fig. 1a shows a schematic structure diagram of a structured light projector 100 according to an embodiment of the present application.

Exemplarily, first referring to FIG. 1 a , the structured light projector 100 includes a light source 110 , a pattern generating unit 120 and a converging lens 130 in sequence. The projection plane 140 is in front of the converging lens 130 . As shown in FIG. 1 a , the light source 110 can project the pattern on the pattern generating unit 120 onto the projection plane 140 through the converging lens 130 . Moreover, the structured light beam emitted by the structured light projector 100 finally forms a specific structured light pattern on the projection plane 140 .

Exemplarily, the light source 110 may be a point light source, for example, may include a single light emitting element. The light emitting element may be a light emitting diode (LED) or a laser diode that emits laser light, or the like. The light emitting element may emit laser light having a wavelength ranging from about 850 nm to about 940 nm, or may emit light having a near-infrared band or a visible band. In one example, the light emitting element may emit red light with a wavelength of 660 nm as structured light. The wavelength of light emitted by the light emitting element is not limited to a specific wavelength. The light source 110 may also include a light emitting element and other components, such as a lampshade, a lamp panel, and a laser emitter capable of emitting structured light.

The pattern generating unit 120 is disposed in front of the light source 110 . The pattern generation unit 120 may generate a specific pattern. After the light emitted by the light source 110 is irradiated on the pattern generating unit 120, a part of it is blocked or the transmission direction is changed, while the other part is transmitted through the pattern generating unit 120 according to a preset angle, thereby forming a desired pattern on the transmission plane 140. spot.

The condensing lens 130 is disposed on a light transmission path between the pattern generating unit 120 and the projection plane 140 . The projection plane 140 can be regarded as an ideal plane, that is, a plane on which the structured light can be projected when no target object is placed. It can be understood that when a target object (such as a finger) is placed on the projection plane 140 , the structured light emitted by the structured light projector 100 is irradiated on the target object, causing a certain optical distortion in the formed structured light pattern. The light passing through the pattern generating unit 120 is converged by the converging lens 130 and reaches the projection plane 140 . The converging lens 130 may include various types of convex lenses, concave lenses or other forms of lenses, or a combination of various lenses. The application is not limited to the converging lens 130 , as long as the converging lens 130 as a whole has a converging effect on light, and can form a clear pattern on the projection plane 140 after converging the light passing through the pattern generating unit 120 .

There is a certain distance between the light source 110, the pattern generating unit 120, and the converging lens 130 on the light transmission path. The distance between the light source 110 and the pattern generating unit 120 may be set as a first preset distance, and the distance between the pattern generating unit 120 and the converging lens 130 may be set as a second preset distance. The first preset distance and the second preset distance can be any suitable distance, both May or may not be equal. The pattern generating unit 120 and the converging lens 130 can also be arranged according to a preset angle. For example, the plane where the pattern generating unit 120 is located can be placed at a preset angle relative to the optical axis of the converging lens 130 . The preset angle may be any suitable angle, such as 90° or 45°. The plane where the pattern generating unit 120 is located may be a plane where a longitudinal section of the pattern generating unit 120 is located. It can be understood that when the thickness of the pattern generating unit 120 is relatively thin, for example, when it is a grating, the pattern generating unit 120 is basically on the same plane as its longitudinal section.

The present application does not limit the first preset distance, the second preset distance and the preset angle here, as long as the configured structured light projector 100 can form a specific structured light pattern on the projection plane 140 .

Exemplarily, the structured light pattern formed by the structured light projector 100 on the projection plane 140 includes a reconstructed structured light pattern and a spliced structured light pattern. Referring to Figs. 1b-1d, schematic diagrams of structured light patterns according to embodiments of the present application are shown. 1b can be regarded as a structured light pattern formed by the structured light beam emitted by the structured light projector 100 in FIG. 1a on the projection plane 140, the structured light pattern is composed of a reconstructed structured light pattern and a stitched structured light pattern. Furthermore, FIG. 1c shows a reconstructed structured light pattern 150 corresponding to the structured light pattern in FIG. 1b , and FIG. 1d shows a mosaic structured light pattern 160 corresponding to the structured light pattern in FIG. 1b .

In the embodiment of the present application, the reconstructed structured light pattern is different from the spliced structured light pattern. Exemplarily, the difference between the reconstructed structured light pattern and the spliced structured light pattern may include that the two patterns are different in shape. Exemplarily, the reconstructed structured light pattern 150 shown in FIG. 1c is different from the mosaic structured light pattern 160 shown in FIG. 1d. The reconstructed structured light pattern 150 is composed of multiple reconstructed structured light units 151 with similar shapes, while the stitched structured light pattern 160 is composed of multiple stitched structured light units 161 with different shapes from the reconstructed structured light units 151 . It can be understood that the reconstructed structured light unit 151 or the spliced structured light unit 161 shown in FIGS. 1c-1d is only one of many. In reconstructing the structured light pattern, the reconstructed structured light unit may have one form or multiple forms. Similarly, in the mosaic structured light pattern, the form of the mosaic structured light unit may be one or multiple.

Exemplarily, in addition to being different in pattern form, the difference between the reconstructed structured light pattern and the spliced structured light pattern may further include different uses of the two. When the structured light projector 100 of the embodiment of the present application is used for image acquisition of a target object, the reconstructed structured light pattern can be used to assist in determining the coordinates of the target object in three-dimensional space and the coordinates in the collected two-dimensional target image The transformation relationship between them, the spliced structured light pattern can be used for the alignment between multiple data to be spliced (which can be two-dimensional unfolded images or three-dimensional target models) for the same target object to assist in the precise splicing of the data to be spliced, and then Acquire high-precision overall data.

Exemplarily, there is no border overlap between the reconstructed structured light unit in the reconstructed structured light pattern and the stitched structured light unit in the stitched structured light pattern. For example, referring to FIG. 1 b , the reconstructed structured light unit 152 in the form of stripes, the square stitched structured light unit 162 , and the circular stitched structured light unit 163 together constitute the structured light pattern in the figure. In Figure 1b, the boundary of each reconstructed structured light unit in stripe form can be regarded as the upper and lower sides of a rectangle, and the square stitched structured light unit 162 and the circular stitched structured light unit 163 are respectively connected with the reconstructed structured light unit in stripe form. 152 up and down No two edges intersect.

The specific functions of reconstructing the structured light pattern and stitching the structured light pattern can be understood with reference to the "image stitching method" described below.

According to the structured light projector of the present application, the structured light emitted by the light source is projected on the surface of the target object through the pattern generating unit and the converging lens to form different reconstructed image patterns and stitched image patterns. A specific reconstructed image pattern can be used to determine the transformation relationship between the 2D target image of the target object and the corresponding data to be stitched, and the stitched image pattern can be used to achieve alignment between the data to be stitched, so a variety of Different structured light patterns help stitching to obtain more accurate overall data.

Exemplarily, the pattern generating unit 120 includes a diffractive optical element or a grating.

Exemplarily, the pattern generation unit 120 may include any existing or future elements or element groups capable of generating specific structured light patterns. In one example, the pattern generating unit may be a diffractive optical element (Diffractive Optical Element, DOE). DOE usually uses a micro-nano etching process to form two-dimensionally distributed diffraction units. Each diffraction unit can have a specific shape, refractive index, etc., so it can fine-tune the phase distribution of structured light such as laser light. The laser light diffracts after passing through each diffraction unit, and interferes at a certain distance to form a specific light intensity distribution, which can form a specific structured light pattern when projected on the projection plane. By way of example and not limitation, a DOE for beam shaping (ie, a DOE for beam shaping) can be used to generate square, polygonal, strip-shaped, ring-shaped, and circular spot patterns. In addition, a diffractive beam splitter can also be used to generate a one-dimensional or two-dimensional array beam so as to form a regular array of scattered points on the surface of the target object.

In another example, the pattern generating unit 120 may be a grating, such as a diffraction grating. Diffraction gratings can periodically modulate the amplitude or phase (or both) of incident light through regular structures. The grating has a pattern. When the light emitted by the light source hits the grating, part of it is blocked and the other part is emitted through the grating, thus forming a light spot with a desired pattern on the surface of the target object.

Exemplarily, the grating can be made of various suitable materials. Exemplarily, the grating may be made of a material with better light transmission, such as a material based on glass or plastic. When the structured light emitted by the light source passes through the grating, it will diffract, so it has certain transmission and divergence effects.

In one example, the grating may include a film on which the reconstructed structured light pattern and the stitched structured light pattern are formed. Exemplarily, the preset pattern may be scaled to a suitable size according to a certain ratio, and then formed on the film by means such as spraying or printing. Exemplarily, referring to FIG. 1b again, if one wants to form a pattern as shown in FIG. Structured light pattern to achieve. Those skilled in the art can easily understand this technical solution, so it will not be repeated here. For the solution where the grating includes film, users can set any desired reconstructed structured light pattern and spliced structured light pattern according to their needs, so it can meet the personalized needs of users.

Exemplarily, the reconstructed structured light unit and the spliced structured light unit can be set to multiple kind of form. Exemplarily, the reconstructed structured light unit includes stripes; and/or, the spliced structured light unit includes scattered points. In an example where the reconstructed structured light unit includes stripes, the stripes may be horizontal stripes, vertical stripes, or oblique stripes. In an example where the spliced structured light unit includes scatter points, the scatter points may include various forms, such as circular scatter points, square scatter points, triangular scatter points, cross-shaped scatter points, and the like. The same spliced structured light pattern may also include a combination of multiple forms of spliced structured light units, such as a combination of square scatter points and cross-shaped scatter points. For example, the spliced structured light unit shown in FIG. 1 b includes square scatter points and circular scatter points.

Exemplarily, in a reconstructed structured light pattern, each reconstructed structured light unit may be arranged according to a fixed rule. For example, stripes of equal width may be arranged at equal intervals, or stripes with different widths may be arranged at different intervals. For example, the reconstructed structured light pattern shown in FIG. 1b is a combination of stripes, and the width of each stripe is not completely the same, and the distance between two adjacent stripes is also not completely the same. Similarly, the spliced structured light units in the spliced structured light pattern can also be arranged according to a fixed rule.

It can be understood that the shape and arrangement of the reconstructed structured light units in the reconstructed structured light pattern and the spliced structured light units in the spliced structured light pattern are determined by the pattern generation unit 120 . For the example where the pattern generation unit 120 is a DOE, the position and angle of the DOE in the light propagation path projected on the projection plane by the DOE, as well as the microstructure of the DOE surface can affect the reconstructed structured light pattern and the stitched structured light projected on the projection plane. pattern.

The stripes have a certain width and length, which is convenient for identification and positioning, and it is also convenient to determine the transformation relationship between the two-dimensional target image of the target object and the corresponding data to be spliced based on it. The distribution of scattered points is relatively dense and the number is large, which is convenient for matching, and then facilitates the alignment of the data to be spliced.

According to the embodiment of the present application, the distribution density of the spliced structured light units is greater than the distribution density of the reconstructed structured light units.

The distribution density of the reconstructed structured light unit may be represented by the distribution density of the marking part on the reconstructed structured light unit (the marking part is at least a part of the reconstructed structured light unit). For example, in the case where the reconstructed structured light unit is a fringe, the distribution density of the midline of each fringe may be used to represent the distribution density of the fringes. Similarly, the distribution density of the spliced structured light unit may be represented by the distribution density of the marking portion on the spliced structured light unit (the marking portion is at least a part of the spliced structured light unit). For example, when the spliced structured light unit is a scatter point, the distribution density of the central points on each scatter point (which usually has a certain area) may be used to represent the distribution density of the scatter points.

The distribution density of the spliced structured light units cannot be too small, otherwise sufficient splicing accuracy cannot be obtained; the distribution density of the spliced structured light units should not be too large, if the distribution density is greater than the deformation error, it is easy to be wrongly matched. For example, referring to FIG. 1b, the distribution density of scattered points is greater than that of stripes.

According to another aspect of the present application, an image acquisition system is also provided. The image acquisition system includes a structured light projector, an illumination system, one or more image acquisition devices and a processing device. The lighting system includes one or more light sources, each light source is used to emit light toward a target collection area, and the target collection area is used to place a three-dimensional target. One or more image acquisition devices are used in the illumination system to target The image of the target acquisition area is acquired while the acquisition area emits light to obtain the target image. The processing device is used for processing the target image collected by one or more image collecting devices.

In the embodiment of the present application, the three-dimensional target may be, for example, the whole human body or a part of the human body, such as human face, fingers, palms, etc., or other objects with three-dimensional structures that need to be collected. It should be understood that the image acquisition system according to the embodiment of the present application may be an image acquisition system for any suitable target object with a three-dimensional shape, such as a human body, a human body part, a three-dimensional still life or an animal, etc., which is not limited here. In one or some embodiments, the target collection area is a handprint collection area, and the three-dimensional target is part or all of the user's hand. For the sake of simplification, the following uses a handprint collection system for collecting fingerprints of fingers as an example for illustration.

Referring to FIG. 2 , FIG. 2 shows a schematic diagram of an image acquisition system 200 and related target objects 400 and target acquisition areas 300 according to an embodiment of the present application. It can be understood that FIG. 2 is a main view effect diagram for the image acquisition system. The image collection system 200 shown in FIG. 2 is a handprint collection system 200 , the target object 400 is a finger 400 , and the target collection area 300 is a handprint collection area 300 .

The structure of the handprint collection system 200 shown in FIG. 2 is only an example and not a limitation to the application, and the handprint collection system 200 according to the embodiment of the application is not limited to the situation shown in FIG. 2 . In the handprint collection system 200 shown in FIG. 2 , there is one structured light projector 100 , three image collection devices, and six light sources in the lighting system. It should be understood that the above number is only an example, and it may be any suitable number, which is not limited in this embodiment of the present application. As another example, the optical axes of the three image acquisition devices 212 , 214 , and 216 shown in FIG. 2 are located in the same plane, and they are respectively facing the target acquisition area 300 from three angles. The optical axis of the image acquisition device 214 located in the middle basically coincides with the central axis 302 of the target acquisition area 300, while the optical axes of the image acquisition devices 212 and 216 on both sides are in a certain clip with the central axis 302 of the target acquisition area 300. horn. Of course, the relative distances and included angles between the three image acquisition devices 212 , 214 , 216 , and between each of them and the target acquisition area 300 can be set as required, and there is no limitation here.

It can be understood that, compared to an illumination light source that is not used for coding but only for illumination, the structured light projector emits structured light for coding.

For example and not limitation, the structured light projector 100 shown in FIG. 2 and the image acquisition device 214 in the middle may be located on different planes to avoid mutual interference. The structured light projector 100 may also be located on the same plane as the image acquisition device 214 . In fact, the structured light projector 100 can be arranged at any suitable position as required, as long as it does not hinder the imaging of the image acquisition device and can illuminate the handprint acquisition area.

According to the image acquisition system of the embodiment of the present application, the structured light emitted by the structured light projector 100 is projected on the surface of the target object to form a reconstructed image pattern corresponding to the reconstructed structured light pattern and a stitched image pattern corresponding to the stitched structured light pattern. Referring to the above description, it can be seen that the reconstructed image pattern can be used to determine the transformation relationship between the two-dimensional target image of the target object and the corresponding data to be stitched, and the stitched image pattern can be used to achieve alignment between the data to be stitched. Therefore, by using the image acquisition system including the structured light projector 100 , alignment and registration between the data to be spliced can be realized, which is helpful for splicing to obtain overall data with higher precision.

In the field of image processing, after collecting images for different parts of a 3D object, in order to obtain more complete information of the 3D object, it is necessary to stitch together the images collected for different parts. At this time, there will be certain splicing problems. The fingerprint identification is taken as an example for description below.

In the process of obtaining rolling fingerprints by non-contact acquisition, it is necessary to splice images collected by multiple cameras to finally obtain a fingerprint image that simulates rolling fingerprints. In the prior art, usually multiple images collected by multiple cameras are directly superimposed. However, due to the possible calibration errors of the camera during the calibration process, and due to many factors such as camera and finger movement during the image acquisition process, there may be errors in the image obtained by directly superimposing the images collected by multiple cameras at different angles. FIG. 3 shows a schematic diagram of a mosaic image obtained by directly superimposing and stitching multiple fingerprint images together in the prior art. Referring to Figure 3, it can be seen that there are obvious seams at the image splicing. Such images will directly affect subsequent fingerprint identification and reduce the accuracy of identification.

In order to at least partly solve the above technical problem, an embodiment of the present application provides an image stitching method. In the image stitching method, the three-dimensional object is irradiated with illumination light to obtain information such as the texture of the three-dimensional object (such as fingerprint information), and at the same time, the three-dimensional object is illuminated with structured light to project a structured light pattern on it. The image pattern formed after projecting the structured light pattern can assist in obtaining the data to be spliced and aligning the data to be spliced, so as to splice the data to be spliced more accurately.

It should be noted that although the above descriptions of the technical problems of the prior art mainly take the mosaic of fingerprint images as an example, the image mosaic method according to the embodiment of the present application can be applied to the mosaic of images acquired for any type of target acquisition . The image stitching method according to the embodiment of the present application may be executed by the processing device 240 in the image acquisition system 200 .

The image stitching method according to the embodiment of the present application will be described below with reference to FIG. 4 . Fig. 4 shows a schematic flowchart of an image mosaic method 400 according to an embodiment of the present application. As shown in FIG. 4 , the image mosaic method 400 includes steps S410 , S420 , S430 , S440 and S450 .

Step S410, acquiring a third target image and a fourth target image, wherein the target image includes a third target image and a fourth target image, and the third target image and the fourth target image are collected by emitting light toward the target collection area while aiming at The target acquisition area acquires images in a manner, the acquisition light includes structured light and illumination light, and the beam of the structured light can form a reconstructed structured light pattern and a spliced structured light pattern; the third target image and the fourth target image correspond to the The acquisition ranges partially overlap.

In this document, terms such as "first", "second", "third", and "fourth" are used for distinction purposes only, and do not indicate order or other special meanings.

The target collection area may be a physical space area with any suitable shape, such as a spherical area, a cubic area, or any other regular or irregular area. Optionally, lighting and image acquisition can be performed on the target acquisition area through the above-mentioned image acquisition system. Exemplarily, the image acquisition system may include one or more image acquisition devices, configured to perform image acquisition on the target acquisition area. Exemplarily but not limitatively, the imaging range of all image acquisition devices in one or more image acquisition devices can cover the entire target acquisition area, so as to facilitate the acquisition of images of the entire target acquisition area. picture. When a three-dimensional object, such as a user's finger or palm, is placed in the target acquisition area, the image acquisition device can acquire an image of the target, such as a fingerprint or palmprint image of the user.

In one embodiment, an image acquisition system may include a single image acquisition device. In this case, optionally, the target images at multiple angles may be collected by rotating and/or moving the image collection device. In another embodiment, the image capture system may include multiple image capture devices. In this case, optionally, multiple image capture devices may be used to capture target images from multiple angles. That is, the optical axes of any two image capture devices among the plurality of image capture devices may form a preset angle with each other, and the preset angle is not equal to 0, so as to collect target images at different angles. Therefore, regardless of using single or multiple image acquisition devices, corresponding target images can be acquired for different parts (ie, different angles) of the three-dimensional target.

Exemplarily, the third target image and the fourth target image may be images acquired by the above-mentioned one or more image capture devices for image capture of different parts of the three-dimensional target. In addition, the third target image and the fourth target image are partially overlapped corresponding to the acquisition range on the target acquisition area. For example, the third target image and the fourth target image may be images captured by two adjacent image capture devices.

In an example, the image acquisition system may include a preset number of cameras arranged in preset positions. Exemplarily, the preset number is 3, and the image acquisition system includes a first camera (hereinafter referred to as C1), a second camera (hereinafter referred to as C2), and a third camera (hereinafter referred to as C3).

Exemplarily, C1 , C2 , and C3 may be arranged at a preset angle on the same plane, or may be arranged at a preset angle on different planes. It can be understood that in order to obtain images of different parts of the three-dimensional object from different positions and angles, the three cameras can be positioned and arranged in a corresponding manner. Those of ordinary skill in the art can easily understand the principle, and details will not be repeated here. Moreover, the images acquired by every two adjacent cameras correspond to the partial overlap of the acquisition range on the target acquisition area. Exemplarily, for C1 and C2 adjacent to each other, the target image I1 and target image I2 acquired at the same time or at different times correspond to the partial overlap of the acquisition range on the target acquisition area; for C2 and C3 adjacent to each other , the target image I2 and the target image I3 respectively acquired at the same time or at different times are also partially overlapped corresponding to the acquisition ranges on the target acquisition area.

The "third target image" and "fourth target image" acquired in step S410 may be any two images to be spliced. For example, when the images collected by C1 and C2 need to be spliced, the third target image and the fourth target image are I1 and I2 respectively; when the images collected by C2 and C3 need to be spliced, the third target image and the fourth target image are respectively I2 and I3.

In addition to the above-mentioned image acquisition device, the image acquisition system may also include a device capable of emitting collected light, such as an illumination system and a structured light projector. Exemplarily, for human body data collection, the image collection system may be a three-dimensional body scanner; for fingerprint collection, the image collection system may be a non-contact 3D fingerprint collection device.

In the embodiment of the present application, the collected light includes structured light and illumination light. Structured Light can be projected onto the surface of the measured object (this article is a three-dimensional target) through a structured light projector Active structure information, such as laser stripes, Gray codes, sinusoidal stripes, etc. Structured light projectors, such as infrared laser emitters, emit specific patterns of near-infrared light. The illuminating light may be visible light meeting preset lighting conditions, so that the third target image and the fourth target image meeting preset quality requirements can be acquired by the image acquisition device.

Exemplarily, the beam of structured light can form a reconstructed structured light pattern and a spliced structured light pattern, and the reconstructed structured light pattern and the spliced structured light pattern are different. Projecting the reconstructed structured light pattern on the surface of the 3D target can be used to reconstruct the body surface shape of the 3D target to obtain the data to be spliced; projecting the spliced structured light pattern on the surface of the 3D target can be used for multiple target images of the 3D target Alignment and splicing synthesis of the corresponding data to be spliced.

It can be understood that after the reconstructed structured light pattern and the spliced structured light pattern are projected onto the 3D target, some optical distortions will usually occur, which can be found and reconstructed in the structured light image corresponding to the collected target image. The light patterns respectively correspond to the image patterns.

Exemplarily, the reconstructed structured light pattern may be a fringe pattern formed by structured light projected by a structured light projector in the handprint collection system. The stripe pattern may include stripes having a preset width. The width of each stripe can be completely the same, such as 0.1 mm wide; or not completely the same, for example, the stripes including three kinds of widths are evenly spaced in sequence. Stripe patterns can also include bright and dark stripes. The reconstructed structured light pattern is composed of a plurality of reconstructed structured light units, and for an example where the reconstructed structured light pattern is a stripe pattern, the reconstructed structured light unit is each stripe. Fig. 5 shows an example of reconstructing a structured light pattern according to an embodiment of the present application. As shown in Figure 5, the reconstructed structured light pattern consists of 15 stripes arranged in a certain order, and each stripe is a reconstructed structured light unit, such as the first dark stripe 510, the first bright stripe 520 and the second dark stripe 530 . In the example of FIG. 5 , the width of the first bright stripe 520 is smaller than the width of the first dark stripe 510 , and the width of the first dark stripe 510 is larger than the width of the second dark stripe 530 .

Exemplarily, the mosaic structured light pattern may be a scatter (or speckle) pattern formed by structured light projected by a structured light projector in an image acquisition system, the speckle pattern includes several speckles, and each speckle may It is a spliced structured light unit. The form of the structured light unit has been described above, and will not be repeated here. Since taking pictures is a projection process, speckle will have unpredictable deformation on the picture. Therefore, some special speckle patterns can be used to ensure that the speckle feature points recognized by multiple cameras will not have errors. For example, setting the speckle in the shape of a cross, and using the cross points as feature points for recognition can help increase the accuracy of speckle recognition.

Fig. 6 shows an example of a structured light pattern including a mosaic structured light pattern according to an embodiment of the present application. Exemplarily, refer to the structured light pattern shown in FIG. 6 , which includes a reconstructed structured light pattern composed of stripes, and a spliced structured light pattern composed of square speckles and circular speckles. As shown in FIG. 6 , both the square speckle 610 and the circular speckle 620 can be regarded as a spliced structured light unit. It can be understood that there are 18 reconstructed structured light units and 488 spliced structured light units included in FIG. 6 .

Exemplarily, the reconstructed structured light unit and the spliced structured light unit may have a certain range overlap. stack. For example, each reconstructed structured light unit may be provided with multiple spliced structured light units. For example, as shown in FIG. 6 , 25 dark speckles are set on the first bright stripe, and 26 bright speckles are set on the second dark stripe.

There is no border overlap between the reconstructed structured light unit and the spliced structured light unit, so that the detection of the reconstructed structured light unit and the spliced structured light unit do not affect each other, so that both can be correctly detected. For example, the rectangular boundary of the second dark fringe shown in FIG. 6 does not overlap with the boundary of any bright speckle within its boundary.

In a specific implementation, the distribution density of the spliced structured light units is greater than the distribution density of the reconstructed structured light units, so that the 3D reconstruction can be performed through the reconstructed structured light units with a smaller distribution density, so as to quickly obtain a 3D model, and then use the distributed The spliced structured light units with high density are finely spliced to ensure splicing accuracy. The distribution density of the spliced structured light units cannot be too small, otherwise sufficient splicing accuracy cannot be obtained; the distribution density of the spliced structured light units cannot be too large, if the distribution density is greater than the deformation error, it is easy to be wrongly matched. For example, referring to FIG. 6 , the distribution density of speckles is greater than that of stripes.

Step S420, determining the first structured light image and the first illumination light image corresponding to the third target image; determining the second structured light image and the second illumination light image corresponding to the fourth target image. Wherein, the first structured light image includes the first reconstructed image pattern corresponding to the reconstructed structured light pattern and the first mosaic image pattern corresponding to the stitched structured light pattern; the second structured light image includes the first reconstructed image pattern corresponding to the reconstructed structured light pattern. The corresponding second reconstructed image pattern and the second stitched image pattern corresponding to the stitched structured light pattern.

In the example where the three-dimensional target is a finger, through the above step S210, n target images collected for the finger can be obtained, and n is 2 for example, then the third target image and the fourth target image collected from different positions/angles can be obtained. target image. And according to the foregoing description, it can be understood that both the third target image and the fourth target image may include the following image information: the structured light image information obtained by emitting structured light to the three-dimensional target from the structured light projector, the three-dimensional target image information transmitted by the lighting system Illuminating light image information acquired by the target emitting illuminating light. In the embodiment of the present application, the structured light image information corresponding to the third target image is represented by the first structured light image, and the illumination light image information corresponding to the third target image is represented by the first illuminated light image; correspondingly, the structured light image information corresponding to the third target image is represented by the first structured light image The second structured light image represents the structured light image information corresponding to the fourth target image, and the second illuminating light image represents the illuminating light image information corresponding to the fourth target image.

Exemplarily, if the color of the structured light belongs to a specific color channel, for example, it is red structured light, the image information of this channel can be directly extracted from any target image, and then the corresponding structured light image can be obtained. If the color of structured light contains multiple color channels, for example, it is yellow structured light, image information of multiple color channels can be extracted from any target image for fusion to obtain a corresponding structured light image.

For example, referring to Figs. 7a-7c, there are shown schematic diagrams of three structured light images corresponding to fingerprint collection at three different angles. Exemplarily, the three structured light images shown in Fig. 7a-7c are obtained by From left to right, it can be obtained by extracting r-channels from the three target images collected by the three cameras C1, C2, and C3 in the non-contact 3D fingerprint collector. The above three target images are C1, C2, and C3 in the The structured light projectors use red light to emit a structured light pattern similar to that shown in FIG. 6 to a three-dimensional target and collect them respectively. It can be seen from Figures 7a-7c that each structured light image contains a reconstructed image pattern corresponding to the reconstructed structured light pattern (bright fringe and dark fringe) shown in FIG. (square speckle and circular speckle) corresponding stitched image pattern.

For example, since there may be many interference factors during the scanning process, there may be some noise or interference areas in the obtained structured light image, which may have adverse effects on subsequent steps, such as increasing the amount of calculation or affecting the accuracy of subsequent measurement and identification. Accuracy, so the acquired structured light image can be optionally de-interferenced. Exemplarily, step S420 may further include: for each target image, acquiring an initial structured light image; performing de-interference processing on the initial structured light image to obtain a de-interferenced structured light image. Subsequent processing is performed based on the de-interferenced structured light image in steps such as step S430. Further explanation will be given below with reference to FIG. 8 . When the subject's nails exist in the collected image, since the subject's nails are not the target area of fingerprint collection, when they inevitably appear in the initial structured light image, it may affect the quality of the final generated fingerprint image. Accuracy, therefore, can optionally be denominated. Exemplarily, a semantic segmentation model (such as a Unet model) may be used to identify nails in the initial structured light image, and remove them as noise points. Fig. 8 shows a schematic diagram of a structured light image after nail removal.

As mentioned above, the illumination light may include blue light and/or green light or light of other colors. Similarly, for the case where the illumination light contains only one color channel, the image information under this channel can be directly extracted from the target image to obtain the corresponding illumination light image. For example, the image information under the blue light channel can be directly extracted to obtain a single-channel illumination light image as a corresponding illumination light image. For the case where the illumination light contains multiple color channels, the image information under each color channel in the multiple color channels can be extracted from the target image first, and the corresponding single-channel illumination light image can be obtained, and then the multiple single-channel illumination The light images are fused to obtain the final illuminated light image.

Exemplarily, step S420 may further include: for each target image, extracting the color channel of the target image to obtain at least one single-channel illumination light image corresponding to at least one color channel; A single-channel illumination light image is fused to obtain an illumination light image corresponding to the target image, wherein at least one color channel is a color channel included in the illumination light. For example, after using C1, C2, and C3 to acquire target images including blue light and green light channels for a three-dimensional target, the image information of blue light and green light channels can be extracted from the target image collected by C1 respectively to obtain the blue light channel illumination light image and The illumination light image of the green light channel, and then fuse the two to obtain the illumination light image corresponding to the target image collected by C1. Similarly, the illumination light image corresponding to the target image collected by C2 and the illumination light image collected by C3 The illumination light image corresponding to the target image.

Step S430: Determine the first transformation relationship based on the reconstruction components contained in the first reconstruction image pattern, and transform the first illumination light image according to the first transformation relationship to obtain the first image to be spliced. Data; the second transformation relationship is determined based on the reconstruction component units contained in the second reconstruction image pattern, and the second illumination light image is transformed according to the second transformation relationship to obtain the second data to be spliced; wherein, the first transformation relationship and the second The transformation relationship is an expansion transformation relationship, the first data to be spliced is the first expansion image, and the second data to be spliced is the second expansion image; or, the transformation corresponding to the first transformation relationship and the second transformation relationship is three-dimensional reconstruction, the first to be spliced The spliced data is the first target model, and the second data to be spliced is the second target model.

Exemplarily, after the reconstructed structured light pattern formed by the structured light, such as the fringe pattern, is projected on the surface of the three-dimensional object, optical distortion may occur. For example, when the reconstructed structured light pattern formed by the structured light emitted by the structured light projector is straight stripes, it can appear arc-like stripes as shown in Figures 7a-7c when projected on the pulp of a finger. Exemplarily, the position and shape of each reconstructed component unit (such as stripes) in the reconstructed image pattern can be analyzed based on the optical principle and the inherent parameters and relative positional relationship of the structured light projector and the image acquisition device, and the The illumination light images corresponding to each target image are respectively transformed to obtain the transformed data to be spliced.

Exemplarily, the first transformation relationship can be determined based on the stripes contained in Figure 7a, and the corresponding illumination light image can be transformed according to the first transformation relationship to obtain the first set of data to be spliced; correspondingly, based on Figure 7b Determine the second transformation relationship based on the stripes contained in , determine the third transformation relationship based on the stripes contained in FIG. 7 c , and further obtain the second group of data to be spliced and the third group of data to be spliced. For example, when it is necessary to stitch the illumination light image under the angle corresponding to Fig. 7a and the illumination light image under the angle corresponding to Fig. 7b, the first transformation relation is the first transformation relation, and the second transformation relation is the second A conversion relationship, the first data to be spliced is the first group of data to be spliced, and the second data to be spliced is the second group of data to be spliced.

In one example, the first transformation relationship and the second transformation relationship are expansion transformation relationships, the first data to be spliced is a first expansion image, and the second data to be spliced is a second expansion image. In this example, the first transformation relationship and the second transformation relationship may be a transformation relationship from a two-dimensional illumination light image to a two-dimensional expanded image. For each target image, each pixel in the two-dimensional illumination light image acquired in step S420 can be directly converted one by one according to the corresponding transformation relationship to obtain new two-dimensional image data. The specific implementation process of this example will be explained in detail below.

In another example, the transformation corresponding to the first transformation relationship and the second transformation relationship is three-dimensional reconstruction, the first data to be spliced is the first target model, and the second data to be spliced is the second target model. In this example, for each target image, three-dimensional reconstruction may be performed on the target image according to the position and shape of its reconstructed constituent units, so as to obtain the corresponding transformation relationship. And the acquired two-dimensional illumination light image is mapped to the reconstructed three-dimensional model according to the transformation relationship, so as to obtain a corresponding three-dimensional target model. The specific implementation process of this example will be explained in detail below.

Step S440, according to the first mapping position of the first mapping and splicing component unit and the second mapping position of the second mapping and splicing component unit, determine the matching relationship between the first mapping and splicing component unit and the second mapping and splicing component unit; wherein, The first map position is used to represent the first map splice group The position of the forming unit, the second mapping position is used to indicate the position of the second mapping and splicing component unit; the first mapping position is equal to the position obtained by performing the operation corresponding to the first transformation relationship on the position of the splicing component unit in the first structured light image , the second mapping position is equal to the position obtained by performing an operation corresponding to the second transformation relationship on the position of the mosaic component unit in the second structured light image.

Exemplarily, refer again to the structured light pattern shown in FIG. 6 . According to the foregoing description, each of the circular speckle and the square speckle can be regarded as a mosaic unit. Since the reconstructed structured light pattern and the spliced structured light pattern formed by the structured light beam can be arranged according to a preset rule, there is also a corresponding positional relationship between the reconstructed component unit and the spliced component unit in the first structured light image. In step S430, the first transformation relationship and the second transformation relationship are obtained. Therefore, the operation corresponding to the first transformation relationship can be performed on the position of the mosaic component unit in the first structured light image, and the corresponding position of the first mapped mosaic component unit can be obtained, that is, the first mapping position; similarly, the second can also be obtained The two-map stitches the constituent units and the second-mapped position.

It is easy to understand that each mapping splicing unit can be regarded as an alignment mark unit for aligning the data to be spliced, and the alignment mark units at approximate positions in the two data to be spliced can be paired, so that in the subsequent splicing process The data to be spliced is spliced based on paired alignment marker units. Exemplarily, the matching relationship between the first mapping mosaic component unit and the second mapping mosaic component unit may include a one-to-one correspondence of matching speckle pairs. By way of example and not limitation, the Unet model can be used to identify the mosaic constituent units in two structured light images corresponding to two adjacent image acquisition devices, and identify where the mosaic constituent units in each structured light image are located. Position coordinates: transform the position coordinates of each stitching component unit according to the expansion transformation relationship, and obtain the position coordinates of each stitching component unit on the image plane where the expanded image is located, thereby obtaining the first mapping of the first mapping stitching component unit The position and the second map are concatenated to form the second map position of the unit. In another example, the two structured light images corresponding to two adjacent image acquisition devices can be converted respectively according to the expansion transformation relationship, and the mapping and splicing constituent units are identified in the converted structured light images to obtain each mapping The position coordinates of the mosaic composition unit on the image plane where the expanded image is located, thereby obtaining the first mapping position of the first mapping mosaic composition unit and the second mapping position of the second mapping mosaic composition unit. It can be understood that the mapping positions obtained by the above two methods are approximately equal, so "equal" in step S440 can be understood as approximately equal. Afterwards, one-to-one matching can be performed on the mapping mosaic constituent units (the first mapping mosaic constituent unit and the second mapping mosaic constituent unit) that meet the preset distance requirement on the image plane where the unfolded image is located, to obtain a matching pair of mapping mosaic constituent units . It should be noted that the first expanded image and the second expanded image are on the same image plane.

In the example where the transformation corresponding to the first transformation relationship and the second transformation relationship is 3D reconstruction, step S440 can map each mosaic component unit in each structured light image to the 3D space one by one (the first object model and the second In the space where the target model is located), the mapping position of each mapping splicing component unit is obtained. For two or more target images, a corresponding number of 3D target models can be obtained, and the 3D target models can be represented by point clouds. For any two models with overlapping point clouds, the mapping positions of the constituent units can be spliced according to the mapping, and the preset distance requirements can be satisfied. The corresponding matching relationship is obtained by splicing the constituent units according to the requested mapping and matching the positions.

Step S450, based on the matching relationship, splicing the first data to be spliced and the second data to be spliced to obtain overall data.

The above step S440 determines the matching relationship between the first mapping mosaic component unit and the second mapping mosaic component unit. For example, if it is determined that the first mapped mosaic component unit and the second mapped mosaic component unit meet a preset distance requirement, they may be regarded as a group of mapped mosaic component unit pairs. For example, the first mapping position is coordinates (2, 9), and the second mapping position is coordinates (3, 9), and the two belong to matching mapping and splicing constituent units. Accordingly, a matching relationship between at least part of the first mapping and splicing constituent units and at least part of the second mapping and splicing constituent units can be determined. Based on the matching relationship, an alignment relationship between at least some of the pixels in the first data to be spliced and at least some of the pixels in the second data to be spliced can be further determined. Thus, the corresponding pixel points in the first data to be stitched and the second data to be stitched can be stitched and synthesized according to the alignment relationship, so as to obtain the overall data.

Exemplarily, based on the same principle, for the case of including the third data to be spliced, on the basis of obtaining the overall data of the first data to be spliced and the second data to be spliced, the obtained overall data can be combined with the third data to be spliced. The spliced data is spliced and synthesized according to the corresponding matching relationship, and finally the overall data about the three are obtained. Of course, it is also possible to select one of them as the reference data according to the matching relationship between the first data to be spliced, the second data to be spliced, and the third data to be spliced respectively (such as selecting the second data to be spliced corresponding to the middle camera) As the benchmark data to be spliced), the three are spliced at one time to obtain the overall data. Those of ordinary skill in the art can understand the implementation principle of the above method, which will not be repeated here.

According to the above technical solution, a plurality of target images collected under structured light and illumination light environment are obtained; according to the reconstruction component units in the structured light image corresponding to the target image, the illumination light image corresponding to the target image is transformed, and each The data to be stitched corresponding to the target image; according to the stitching components in the structured light image, multiple data to be stitched are accurately stitched and synthesized to obtain the synthesized overall data. Different from the existing technology, the image mosaic method of the embodiment of the present application uses different components in the structured light image to act on different processing functions, one part is used for data form transformation, and the other part is used for precise alignment between data . The splicing accuracy of the overall data of the target object obtained after processing through this scheme is higher, the splicing effect is better, and it is more friendly to the derived field where the data is applied. For example, the simulated scrolling fingerprint image obtained through this scheme can be compatible with the fingerprint image collected by the traditional printing method, which can greatly improve the accuracy of fingerprint recognition, and then can significantly improve the user experience.

Exemplarily, in step S430, the first transformation relationship and the second transformation relationship are expansion transformation relationships, the first data to be spliced is a first expansion image, and the second data to be spliced is a second expansion image. Determining the first transformation relationship based on the reconstructed component units contained in the first reconstructed image pattern may include: performing three-dimensional reconstruction according to the positions of the reconstructed component units contained in the first reconstructed image pattern to obtain a first model; according to the first reconstructed image pattern The position of the reconstructed constituent unit contained in and the first model obtain the first first-order transformation relationship; expand the first model along the common coordinate axis to obtain the first The unfolded position of each point on the model; the first second-order transformation relationship is obtained according to the first model and the unfolded positions of each point on the first model; the first transformation relationship is obtained according to the first first-order transformation relationship and the first second-order transformation relationship. In addition, determining the second transformation relationship based on the reconstructed constituent units contained in the second reconstructed image pattern includes: performing three-dimensional reconstruction according to the positions of the reconstructed constituent units contained in the second reconstructed image pattern to obtain a second model, and according to the second reconstructed image The position of the reconstructed unit contained in the pattern and the second model obtain the second first-order transformation relationship; expand the second model along the common coordinate axis to obtain the expanded position of each point on the second model; according to the second model and the second model The expansion position of each point above is used to obtain the second second-order transformation relationship; the second transformation relationship is obtained according to the second first-order transformation relationship and the second second-order transformation relationship.

The implementation method of the above steps will be specifically explained below by taking FIG. 8 as an example of the first structured light image.

The reconstruction unit in Fig. 8 is a number of bright and dark stripes with different widths, and the first model can be obtained by performing three-dimensional reconstruction according to the position of each stripe in the image coordinate system. Exemplarily, FIG. 8 shows a plurality of stripes with different positions and shapes, and each stripe may be marked sequentially according to a preset rule. For example, the thinnest stripe 810 can be marked as i=0, the remaining stripes can be marked as i=-1,-2,-3,...,-6 upwards, and the rest of the stripes can be marked as i= 1, 2, 3, ..., 11.

Exemplarily, the structured light image can be input to the neural network used to detect the reconstruction components, and the labels of the reconstruction components included in the structured light image and the positions of the reconstruction components in the image coordinate system can be obtained. Exemplarily, the position of the reconstructed component unit in the image coordinate system may be represented by the coordinates of several points distributed with a preset point density on the centerline of the reconstructed component unit in the image coordinate system. It can be understood that the position of the reconstruction unit can also be represented by other points such as the upper and lower edge points of the reconstruction unit. However, when the size of the detected reconstruction unit changes due to the change of the pixel value of the structured light image (the edge tend to change with it), it is more robust to use the points on the midline. It is easy to understand that the number of points acquired on the center line of each stripe may not be exactly the same. Let the coordinates of a point on the midline of each reconstruction constituent unit be (x, y).

Exemplarily, three-dimensional reconstruction can be performed on each reconstruction component unit through the projection matrix of the camera and the light surface parameters of the structured light. Exemplarily, for the case where the reconstruction unit is a fringe, the three-dimensional coordinates (u, v, w) of each point on the centerline of each fringe can be obtained.

For example, for a stripe with i=1, the following formula (1) exists:

Among them, p represents the projection matrix of the camera, as a known quantity; (u, v, w) represents the 3D coordinates of the point on the fringe center line for 3D reconstruction, as an unknown quantity; (x, y) represents the reconstruction in the structured light image The coordinates of the point on the center line of the fringe with i=1 on the structured light pattern are taken as a known quantity; α is a distance coefficient and taken as an unknown quantity.

For the case where p is a 3*3 matrix, it is easy for those of ordinary skill in the art to understand, from the above formula Formula (1) can get 3 equations, and can further solve the unknown quantities u, v, w, α among them.

Exemplarily, before the target image is collected, the normal vector of the light plane where each reconstruction component (for example, fringe) is located may be obtained through a calibration process for each camera. Fig. 9 shows a simple schematic diagram of collecting structured light stripes by a camera. FIG. 9 shows the first optical surface 910 where the stripe i=1 is located and the second optical surface 920 where the stripe i=2 is located. Based on the results of camera calibration, the corresponding relationship between the three-dimensional coordinates of each stripe i=n in the structured light image and the normal vector of the light surface where it is located can be obtained, and the fourth point corresponding to the point on the center line of each stripe can be obtained. equations, such as the following formula (2).

in, is the normal vector of the light surface where any fringe is located, which is taken as a known quantity.

From this, a total of 4 equations can be obtained for the points of the center line of each fringe. By solving the above four equations, the three-dimensional coordinates (u, v, w) of each point on the centerline of each stripe after three-dimensional reconstruction can be obtained.

Exemplarily, according to the two-dimensional image coordinates (x, y) of each point on the centerline of each stripe in the first structured light image and the three-dimensional coordinates (u, v, w) corresponding to the three-dimensional reconstruction, each point on the centerline of the stripe can be solved. The corresponding relationship between the 2D image coordinates of the points and the 3D coordinates corresponding to the 3D reconstruction. The corresponding relationship is a first-order transformation relationship, which can be a three-dimensional reconstruction difference function, and is represented by the following formula (3):
(u, v, w) = f(x, y) Formula (3)

Exemplarily, according to the above formula (3), the three-dimensional coordinates of each point in the first structured light image (including other points except the point on the fringe center line) can be further obtained. In this way, in the three-dimensional space, a visualized three-dimensional point cloud model as shown in FIG. 10 , that is, the first model can be obtained.

Similarly, referring to the above solution, the second model corresponding to the fourth target image of the same three-dimensional target, and another target image (such as the fifth target image) different from the third target image and the fourth target image can be obtained. The corresponding third model and so on, as well as the first-order transformation relationship corresponding to each model, will not be repeated here.

Exemplarily, after the first model and the first first-order transformation relationship are obtained through the above steps, the first model can be further expanded. The three-dimensional surface shape of the finger can be regarded as a cylinder-like body, the central axis of the cylinder can be found, and the side surface of the cylinder can be expanded according to the central axis. In the embodiment of the present application, the common coordinate axes of the multiple 3D models for the 3D object may be found first, and then unfolded according to the common coordinate axes.

Exemplarily, before unfolding the first model along the common coordinate axis to obtain the unfolded position of each point on the first model, the method 400 may further include: according to the corresponding reconstruction in the first model and the second model, the same The 3D points of the reconstructed constituent units form a 3D point group, and the 3D coordinates of the 3D point group are fused to obtain the fused coordinates; the common coordinate axis is determined according to the fused coordinates of each reconstructed constituent unit.

Exemplarily, merging the 3D coordinates of the 3D point group to obtain the fused coordinates and determining the common coordinate axis according to the fused coordinates of each reconstructed constituent unit includes: according to the points on the midline of each reconstructed constituent unit (such as a stripe) in the 3D point group The three-dimensional coordinates of each reconstruction unit are determined to determine the fusion coordinates of each reconstruction unit; the method of principal component analysis is used to determine the corresponding common coordinate axis according to the fusion coordinates of each reconstruction unit.

Exemplarily, among the three models obtained in the above steps, the three-dimensional coordinates of each point on the midline of each reconstruction component unit can be searched respectively. For example, extracting the midline corresponding to the i=5 stripes in the three models, it can be understood that there are multiple overlapping points between any two of the three midlines. Points on the three median lines corresponding to stripe i=5 form a three-dimensional point group of stripe i=5.

A deduplication operation can be performed on the points located on the midline of the same reconstruction unit to obtain the fusion coordinates of the reconstruction unit. Exemplarily, the coordinates of the points in the overlapping areas in each two models may be fused by means of averaging. For example, on the overlapping midline, each model has 10 points, and the 10 points on the two models can be associated one by one according to the closest relationship, that is, 10 three-dimensional point pairs can be obtained, and each The midpoints of three-dimensional point pairs, and each midpoint is used as the fused point of the overlapping area. For fringe i=5, the fused points in the overlapping area plus the original points in the non-overlapping area of the three models on the fringe can form the deduplicated point. The coordinates of the deduplicated points of the stripe i=5 can be regarded as the fused coordinates of the stripe i=5.

In another example, deduplication can also be performed with the intersection center point of each overlapping area as the boundary, for example, for AA ₁ and B ₁ B in Figure 11a, the overlapping area B ₁ can be removed with the intersection point O as the boundary O and points in the overlapping area OA ₁ , get the deduplicated AB shown in Figure 11b. The coordinates of the points on AB after deduplication are fusion coordinates.

The de-duplication operation can alleviate the problem of inaccurate determination of the common coordinate axis when the finger is placed off-center, thereby improving the quality of image processing.

As mentioned above, after the deduplication operation is performed on the points on the midline of the same reconstruction unit in multiple 3D models, the deduplicated combined midline (the result of the combination of three midlines) can be obtained, and the points on the combined midline The coordinates are the fused coordinates corresponding to the reconstructed constituent units. The three-dimensional coordinates of the points on the preset position on the combined center line after deduplication can be extracted, and the corresponding common coordinate axes can be determined by the method of principal component analysis. The points at the preset positions may include, for example, the left and right endpoints of the combined center line after deduplication. Exemplarily, the three-dimensional coordinates of the endpoints at both ends of each deduplicated combined median line may be extracted at one time, and input into the principal component analysis algorithm model to obtain a common coordinate axis. Exemplarily, the point at the preset position may also include the left and right endpoints of the combined midline after deduplication and the midpoint of the combined midline, and input the three-dimensional coordinates of all three points on the combined midline into principal component analysis In the algorithm model, the common coordinate axes are obtained. Of course, other suitable points on the combined median line after deduplication can also be selected for analysis to obtain the common coordinate axis.

After obtaining the common coordinate axis, each three-dimensional model can be expanded along the common coordinate axis. First, the three-dimensional Cartesian coordinate system can be re-established based on the common coordinate axis, each The new coordinates of the point are denoted as (u', v', w'). Referring to FIG. 12 below, FIG. 12 is a schematic diagram of a method for unfolding a three-dimensional fingerprint model. It can be understood that in an ideal state, the coordinates of the w' axis of each point on the center line of each stripe in the new three-dimensional coordinate system are equal. Based on this principle, each stripe can be unfolded sequentially along the common coordinate axis.

There are many ways to expand, and only one specific way to expand is used as an example for illustration.

Exemplarily, a plane l can be constructed through the v' axis and the w' axis, and multiple intersection points of the plane l and each reconstruction component unit can be obtained. For example, in FIG. 12 , the stripe center line with i=0 intersects the plane l at point O ₁ (u' _O1 , v' _O1 , w' _O1 ), and it is easy to understand that u' _O1 =0. Exemplarily, it is also possible to make a plane l' perpendicular to the v' axis through point _O1 , and use this plane as the datum plane for the expansion of each point on the first model, that is, all the points after the first model is expanded along the common coordinate axis All points can fall on this datum. Since the coordinates of the v' axis of each point are equal on this datum plane, the coordinates of each point after expansion can be represented by (p, q).

Exemplarily, for the point O ₁ in FIG. 12 , the expanded position is still the same as the point O ₁ , and its coordinates can be expressed as (p _O1 , q _O1 ). It can be understood that p _{O1 =} u' _O1 , q _{O1 =} w' _O1 . On the fringe centerline where i=0, take a point A ₁ to the right along point O ₁ , and find the position A ₁ '(p ₁ , q ₁ ) of the corresponding point of A ₁ in the development datum plane. It is easy to understand that q ₁ =q _O1 , and when A ₁ is infinitely close to O ₁ , the length of the arc O ₁ A ₁ is approximately equal to the straight-line distance p ₁ of O ₁ A ₁ . Therefore, the following calculation formula (4) can be obtained:

Wherein, α represents a pixel coefficient, and α>1. The pixel coefficient can be set according to the pixel density of the expanded image.

By analogy, the position of each point on the fringe center line with i=0 in the development datum plane can be calculated in turn, and then the position coordinates (p, q) of all points in the first model on the development datum plane can be obtained.

Moreover, according to the three-dimensional coordinates (u, v, w) of each point in the first model and the corresponding unfolded position coordinates (p, q), the corresponding relationship between the two can be obtained, and the corresponding relationship is the second-order transformation relationship , which can be expressed by the following formula (5):
(p, q) = g(u, v, w) formula (5)

Exemplarily, the first transformation relationship includes a first-order transformation relationship and a second-order transformation relationship. Further, according to formula (3) and formula (5), the corresponding relationship between each point on the first structured light image and the unfolded position point can be obtained. This corresponding relationship is the first transformation relationship, which can be expressed by the following formula (6):
(p, q) = F(x, y) formula (6)

Based on the same principle, the second first-order transformation relationship, the second second-order transformation relationship, and the second transformation relationship corresponding to the fourth target image can also be obtained. Those of ordinary skill in the art can easily understand this implementation method, and will not repeat them here. .

Exemplarily, in step S430, after the first transformation relationship is acquired, the pixel points on the first illumination light image may be directly transformed according to the first transformation relationship to obtain the first data to be spliced. It can be understood that the first illumination light image may contain several pixels, for example, 3072*2048 pixels. The position coordinates of each pixel point can be substituted into the above formula (6) to obtain the unfolded position coordinates of each pixel point, and accordingly transform the pixel values of each pixel point to the unfolded position one by one, thus obtaining the Expand the image corresponding to the image to obtain the first data to be spliced. The acquisition method of the second data to be spliced is similar to that of the first data to be spliced, and will not be repeated here.

The above technical solution can directly convert the two-dimensional illumination light image into a two-dimensional unfolded image according to the calculated transformation relationship. Image processing efficiency.

In another example, the first transformation relationship and the second transformation relationship are expansion transformation relationships, the first data to be spliced is the first expansion image, and the second data to be spliced is the second expansion image; based on the first reconstructed image pattern contains Determining the first transformation relationship of the reconstructed constituent units includes: performing three-dimensional reconstruction according to the positions of the reconstructed constituent units contained in the first reconstructed image pattern to obtain the first model, and according to the positions and The first-order transformation relationship is obtained from the first model; unfold the first model along the common coordinate axis to obtain the unfolded position of each point on the first model; obtain the first-two An order transformation relationship; wherein, the first transformation relationship includes a first first-order transformation relationship and a first second-order transformation relationship. In addition, determining the second transformation relationship based on the reconstructed constituent units contained in the second reconstructed image pattern includes: performing three-dimensional reconstruction according to the positions of the reconstructed constituent units contained in the second reconstructed image pattern to obtain a second model, and according to the second reconstructed image The position of the reconstructed unit contained in the pattern and the second model obtain the second first-order transformation relationship; expand the second model along the common coordinate axis to obtain the expanded position of each point on the second model; according to the second model and the second model The expanded position of each point above obtains the second second-order transformation relationship; wherein, the second transformation relationship includes the second first-order transformation relationship and the second second-order transformation relationship.

In this case, transforming the first illumination light image according to the first transformation relationship to obtain the first data to be spliced may include: performing a first-order transformation on the first illumination light image according to the first first-order transformation relationship to obtain the first A three-dimensional model corresponding to the illumination light image; performing a second-order transformation on the three-dimensional model corresponding to the first illumination light image according to the first second-order transformation relationship to obtain a first expanded image corresponding to the first illumination light image. Transforming the second illumination light image according to the second transformation relationship to obtain the second data to be spliced may include: performing a first-order transformation on the second illumination light image according to the second first-order transformation relationship to obtain the data corresponding to the second illumination light image. the three-dimensional model; according to the second second-order transformation relationship, the second-order transformation is performed on the three-dimensional model corresponding to the second illumination light image to obtain a second expanded image corresponding to the second illumination light image.

The foregoing embodiment of calculating the first transformation relationship by using the first first-order transformation relationship and the first second-order transformation relationship is merely an example rather than a limitation to the present application. In another embodiment, the first transformation relationship It may include a first first-order transformation relationship and a first second-order transformation relationship. In this case, the first first-order transformation relationship and the first second-order transformation relationship can also be determined by obtaining the first model through the above-mentioned three-dimensional reconstruction and unfolding the first model. But at this time, it is not necessary to calculate the formula (6) and directly transform the pixels on the first illumination light image into the first expanded image based on the formula (6), but firstly based on the first first-order transformation relationship, that is, the formula ( 3), transform the pixel points on the first illumination light image to three-dimensional space, and then based on the first second-order transformation relationship, that is, formula (5), continue to transform the pixel points on the first illumination light image from three-dimensional space to two-dimensional space Dimensionally expanded image. That is, the pixel points on the first illumination light image can be transformed to the two-dimensional expanded image by two indirect transformations to obtain the first expanded image. The acquisition method of the second expanded image is similar to that of the first expanded image, which will not be repeated herein.

The intermediate process of this solution can also show the user the 3D reconstruction effect corresponding to the illumination light image (such as a fingerprint image), which is convenient for the user to check and modify, and can also provide a variety of visual images to meet the various needs of the user.

Exemplarily, in step S430, the transformation corresponding to the first transformation relationship and the second transformation relationship is a three-dimensional reconstruction, the first data to be spliced is the first target model, and the second data to be spliced is the second target model; based on the first reconstruction Determining the first transformation relationship for the reconstructed constituent units contained in the image pattern includes: performing three-dimensional reconstruction according to the positions of the reconstructed constituent units contained in the first reconstructed image pattern to obtain a first model; The position of the constituent unit and the first model obtain the first transformation relationship; transform the first illumination light image according to the first transformation relationship to obtain the first data to be spliced, including: performing a transformation on the first illumination light image according to the first transformation relationship order transformation to obtain the first target model; determining the second transformation relationship based on the reconstructed constituent units contained in the second reconstructed image pattern, including: performing three-dimensional reconstruction according to the positions of the reconstructed constituent units contained in the second reconstructed image pattern, to obtain the first The second model; obtain a second transformation relationship according to the position of the reconstruction component unit contained in the second reconstruction image pattern and the second model; transform the second illumination light image according to the second transformation relationship to obtain the second data to be spliced, including: A first-order transformation is performed on the second illumination light image according to the second transformation relationship to obtain a second object model.

Different from the example described above where the illumination light image is transformed into an unfolded image, in this example, the unwrapping process for the 3D model (namely the first model and the second model) may not be performed, but the illumination light image containing The pixel data is also transformed into three-dimensional space, resulting in a first object model and a second object model. Then, the first object model and the second object model can be directly spliced. For example, after obtaining the above-mentioned first model and formula (3), directly map the pixel values of each pixel point of the first illumination light image to the three-dimensional space where the first model is located according to formula (3), and obtain the A first object model of pixel information in an illumination light image. A similar transformation can also be performed on the second illumination light image to obtain a second target model.

Exemplarily, before step S440, the image mosaic method 400 may further include: detecting the first structured light image to obtain the position of the mosaic component unit in the first structured light image; Perform the operation corresponding to the first transformation relationship on the position to obtain the first mapping position; detect the second structured light image to obtain the stitching in the second structured light image The position of the component unit; the operation corresponding to the second transformation relationship is performed on the position of the spliced component unit in the second structured light image to obtain the second mapping position.

Exemplarily, the following description will be made by taking FIG. 7a as an example of the first structured light image and FIG. 7b as an example. The Unet model can be used to detect and identify each circular speckle and square speckle in Fig. 7a and Fig. 7b respectively, and obtain the position coordinates of each circular speckle and square speckle. Then, the position coordinates of each speckle can be substituted into the aforementioned formula (3) or formula (6), to obtain the position coordinates of each speckle in the image plane where the three-dimensional model or expanded image is located.

In this embodiment, the position of the mosaic component unit is firstly detected from the structured light image, and then only the position information is transformed to obtain the position of the corresponding mapped mosaic component unit. This solution only needs to perform position calculations, and does not need to map the pixel values of the structured light image to the image plane where the unfolded image is located, so the amount of calculation is relatively small.

Exemplarily, the image stitching method 400 further includes: performing an operation corresponding to the first transformation relationship on the first structured light image to obtain a first mapped structured light image; performing mapping and stitching composition unit detection on the first mapped structured light image to obtain a first mapped structured light image A mapping position; performing operations corresponding to the second transformation relationship on the second structured light image to obtain a second mapped structured light image; performing mapping and splicing composition unit detection on the second mapped structured light image to obtain a second mapping position.

Exemplarily, since the structured light image includes both the reconstructed structured light pattern information and the spliced structured light information, the first structured light image and the second structured light image can also be transformed according to the obtained first transformation relationship and the second transformation relationship Carry out image conversion, map the mosaic structured light image information together to a new structured light image, and then detect the position of each mapped mosaic component unit on the mapped structured light image. Referring again to Figures 7a-7c, the coordinates of each pixel in the three structured light images in Figures 7a-7c can be substituted into formula (3) or formula (6), and three expansions including stitched structured light patterns can be obtained The graph or three 3D models including spliced structured light patterns, and the pixel values of the pixel points on the structured light image are also mapped to the corresponding expanded graph (mapped structured light image) and 3D model. Then speckle detection is performed on the unfolded image or the 3D model to obtain the position coordinates of each speckle, and then the mapping position of the constituent units of the mapping splicing can be obtained.

Exemplarily, the transformation corresponding to the first transformation relationship and the second transformation relationship is three-dimensional reconstruction, the first data to be spliced is the first target model, and the second data to be spliced is the second target model; the image stitching method 400 may also include the steps S460. Expand the overall data along the common coordinate axis to obtain an overall expanded image.

Exemplarily, for the case where the unfolding process for the 3D model is not performed in step S430, the first object model and the second object model can be spliced through steps S440 and S450 to obtain the pixel information including the illumination light image The three-dimensional overall data, and then expand the three-dimensional overall data. The principle of the unfolding process in step S460 is similar to the above method of unfolding the 3D model to a 2D plane. Each pixel in the overall data can be unfolded according to the second-order transformation relationship at one time to obtain the overall unfolded image.

Exemplarily, step S440 includes at least one of the following: a first search operation, a second search operation operation and inspection operations; where,

The first search operation includes: for any specific first mapping mosaic component unit, searching the second mapping mosaic component unit closest to the specific first mapping mosaic component unit from the second mapping mosaic component unit as the specific first mapping mosaic component unit The second mapping of unit matching is spliced to form a unit;

The second search operation includes: in the first mapping and splicing composition unit, use the first current mapping and splicing composition unit as the first starting point to search for the target first mapping and splicing composition unit that meets the relative position condition with the first starting point; In the composition unit, use the second current mapping and splicing composition unit as the second starting point to search for the target second mapping and splicing composition unit that meets the relative position condition with the starting point, and use the target second mapping and splicing composition unit as the target first mapping and splicing composition unit to match The second mapping splicing composition unit; wherein, the first current mapping splicing composition unit matches the second current mapping splicing composition unit;

The checking operation includes: calculating the distance between the mapping positions of the first mapped mosaic unit that matches each other and the mapping position of the second mapped mosaic unit to obtain a matching error; matching the first mapped mosaic unit that matches each other and the The second mapping and splicing component unit is determined as the final matching first mapping and splicing component unit and the second mapping and splicing component unit; the multiple mapping and splicing component unit pairs included in the matching relationship are the final matching multiple mapping and splicing component unit pairs.

Exemplarily, based on the mapping position of each mapping and mosaic component unit, such as the coordinate value, the mapping and mosaic component units corresponding to the two target images may be searched for matching with a preset distance requirement. The preset distance requirement may be, for example, the shortest distance. The mapping mosaic constituent unit may be, for example, a bright/dark circular speckle or a square speckle located on each stripe as shown in FIG. 6 . For example only, the structured light pattern shown in FIG. 6 can be projected on the finger to be collected to obtain two target images, and the two target images can be input into the Unet model, and the position coordinates of each speckle can be identified, and the By substituting the coordinates into formula (3) or formula (6) in the above example, each speckle can be mapped to a corresponding unfolded image or a three-dimensional model.

Exemplarily, the first search operation may include the following operations. Among the first mapped mosaic units determined based on the position mapping of the mosaic units in the first structured light image, find any specific first mapped mosaic unit, for example, corresponding to the bright square speckle 610 in FIG. 6 The first mapped speckle. Then, from the second mapping mosaic constituent units determined based on the position mapping of the mosaic constituent units in the second structured light image, the second mapping mosaic constituent unit with the closest distance to the specific first map mosaic constituent unit is searched, for example, the distance from the first The closest bright square speckle (second mapped speckle) of the speckle is mapped, and its coordinates are recorded. Subsequently, it may be judged whether the above two mapping mosaic units (the first mapping speckle and the second mapping speckle) match. Exemplarily and not limitatively, whether two map stitching constituent units match may be determined by judging whether the distance between them meets a first preset distance requirement, for example, whether it is smaller than a first predetermined distance threshold. The first predetermined distance threshold may be any suitable value. The first mapping mosaic component unit and the second mapping mosaic component unit that meet the first preset distance requirement may be considered as matching. Exemplarily, it can be By analogy, multiple pairs of matching mapping splicing constituent units are found.

In addition, a second lookup operation may also be performed. The first lookup operation and the second lookup operation can coexist. In the second search operation, the first starting point may be the first mapping combination unit (ie, the first current mapping combination unit) in any pair of currently matched mapping combination units. For the first starting point, it is possible to search for a target first mapping splicing component unit that satisfies a relative position condition with the first starting point. The relative position condition may include that the distance from the first starting point satisfies a second preset distance requirement, or is the closest to the first starting point, or is the closest to the first starting point along a predetermined direction, and so on. The second preset distance requirement may be the same as or different from the above-mentioned first preset distance requirement. For example, the second preset distance requirement may be less than or equal to a second predetermined distance threshold, and the second predetermined distance threshold may be any suitable value. For example, the speckle located at the center of any fringe (it can be understood that it refers to the first mapped speckle located at the center) may be used as the initial first starting point. A second mapped speckle that matches the first starting point at this time is found from each of the second mapped speckles (a manner of searching for a matched mapped speckle may optionally be a first search operation). Subsequently, the next first mapped speckle may continue to be found from the first mapped speckle located at the center, and the next first mapped speckle may be the closest to the first mapped speckle located at the center along a predetermined direction The first mapped speckle, and then performs speckle matching based on the found first mapped speckle, and finds a second mapped speckle that matches the first mapped speckle. Subsequently, using the newly matched first mapped speckle as a new first starting point, continue to find the next first mapped speckle and continue matching. The above operations can be performed cyclically.

In addition, a third lookup operation may also be performed. According to the first mapping position of the above-mentioned first mapping and splicing component unit and the second mapping position of the above-mentioned second mapping and splicing component unit, perform global matching on each first mapping and splicing component unit and each first mapping and splicing component unit, so that the overall matching The error is the smallest; the above-mentioned overall matching error is the sum of the mapping position differences between the first mapping and splicing constituent units and the second mapping and splicing constituent units having a matching relationship. For example, there are 5 first mapping mosaic units and 4 second mapping mosaic units, when the No. 1-3 first mapping mosaic units and No. 1-3 second mapping mosaic units match respectively, No. 1 The square of the distance between the mapping positions of the first mapping mosaic constituent unit and the No. 1 second mapping mosaic constituent unit and the distance between the mapping positions of the No. 2 first mapping mosaic constituent unit and the No. 2 second mapping mosaic constituent unit, Add the squares of the distances between the mapping positions of the No. 3 first mapping mosaic unit and the No. 3 second mapping mosaic unit to obtain the overall matching error; when the No. 1-3 first mapping mosaic unit and No. 2-4 No. When the two mapping mosaic units are matched respectively, the square of the distance between the mapping positions of the No. 1 first mapping mosaic unit and the No. 2 second mapping mosaic unit and the No. 2 first mapping mosaic unit and No. 3 second mapping The sum of the square of the distance between the mapping positions of the mosaic constituent units and the square of the distance between the mapping positions of the No. 3 first mapping mosaic constituent unit and the No. 4 second mapping mosaic constituent unit is added to obtain the overall matching error. The matching relationship when the overall matching error is found to be the smallest is the determined matching relationship between the first mapping and splicing component unit and the second mapping and splicing component unit.

Further, multiple matching speckle pairs of markers can also be checked, for example, one can calculate The relative distance value between each matching speckle pair. By way of example and not limitation, the average value M and variance μ of multiple relative distance values may be calculated, and a threshold value of the relative distance, such as M±μ, may be set, and speckle pairs exceeding the range of the threshold value may be eliminated. Through the above checking operation, the matching error can be checked to obtain a more accurate matching result. Exemplarily, step S450 may include the following steps S451, S452, and S453: step S451, based on the matching relationship, determine the position correspondence between the first data to be spliced and the second data to be spliced; step S452, based on the position correspondence, One of the first data to be spliced and the second data to be spliced is spliced and transformed to obtain transformed data to be spliced; step S453, the untransformed untransformed data to be spliced in the first data to be spliced and the second data to be spliced is not transformed. Splicing the data and transforming the data to be spliced is spliced to obtain the overall data.

The following takes the first data to be spliced and the second data to be spliced as expanded images as an example for specific description.

Exemplarily, according to step S440, a plurality of speckle pairs and position coordinates of each speckle pair on the first data to be spliced and the second data to be spliced respectively can be obtained. Speckle pairs such as D1*D2, where D1(2,3), D2(2.1,3); E1*E2, where E1(3,4), D2(3.2,4); F1*F2, where F1(3 ,5), F2(3.2,5). Exemplarily, a transformation function about the first data to be spliced and the second data to be spliced can be constructed according to the coordinate relationship of the above-mentioned three speckle pairs or more speckle pairs (for example, by TPS thin-plate spline interpolation method). The splicing transformation mainly makes one of the first data to be spliced and the second data to be spliced be adjusted such as scaling after transformation, so as to keep the same size or substantially the same size as the corresponding area in the other data to be spliced.

Exemplarily, each pixel point of the first data to be spliced or the second data to be spliced can be transformed according to the above transformation function to obtain transformed data to be spliced. In the following, it is assumed that the first data to be spliced is spliced and transformed. It is easy to understand that the converted data to be spliced and the second data to be spliced are basically the same in size at the area corresponding to the position. On this basis, the splicing and fusion of the two can quickly obtain the overall data with less error. In another example, the second data to be spliced may also first be spliced and transformed, the principle of which is the same as that of the above solution, and the user may set it as required.

The splicing example in which the first data to be spliced or the second data to be spliced is a three-dimensional object model is similar to the above method, which can be understood by those skilled in the art and will not be repeated here.

Exemplarily, the first transformation relationship and the second transformation relationship are expansion transformation relations, the first data to be spliced is the first expansion image, the second data to be spliced is the second expansion image; the untransformed data to be spliced is the untransformed expansion image ; Transform the data to be spliced into a transformed expanded image; splicing the untransformed untransformed data to be spliced and the converted data to be spliced in the first data to be spliced and the second data to be spliced to obtain the overall data, including: based on untransformed Determining pixel values in the common image area of the overall data based on pixel values in the common image area in the expanded image and/or transforming pixel values in the common image area in the expanded image; based on the pixel values in the common image area in the first expanded image Based on the pixel values in the area, determine the pixel values of the overall data located in the first non-common image area; based on the pixel values located in the non-public image area in the second expanded image, determine the overall The pixel values of the data that lie within the second non-common image area. Wherein, the common image area is the area where the matching mapping and stitching constituent units of the first mapping stitching composition unit and the second mapping stitching composition unit are located, and the first non-common image region is the first mapping stitching composition unit not matched with the second mapping stitching composition unit. The area where the mapping and stitching constituent units are located; the second non-common image area is the area where the second mapping and stitching constituent units that are not matched with the first mapping and stitching constituent units are located.

For example, refer to FIG. 13 , which shows a schematic diagram of stitching two images. As shown in FIG. 13 , the first expanded image may be transformed first to obtain a transformed expanded image 1310 . It can be understood that since the first non-public image area 1330 is completely transformed from the first expanded image, the pixel value of this area can be equal to the corresponding pixel value of the first expanded image; and the second non-public image area 1340 is the second A part of the expanded image 1320 does not include any information of the first expanded image, so the pixel value of this area may be equal to the corresponding pixel value of the second expanded image. The common image area 1350 can be the overlapping area of the transformed expanded image 1310 and the second expanded image 1320, so the pixel value of this area can adopt the pixel value of either of the two, or can be determined based on both, for example, average way.

The above-mentioned method for determining the pixel value of the overall data is simple and easy to operate, and has a small amount of calculation, which can be realized by directly loading corresponding logic operations in the image processing model.

Exemplarily, the overall data includes a common area, a first non-common area and a second non-common area, and the common area is the area where the matching mapping and mosaic constituent units of the first mapping mosaic constituent unit and the second mapping mosaic constituent unit are located, The first non-common area is the area where the first mapping and splicing constituent units that are not matched with the second mapping splicing constituent unit are located, and the second non-common area is the area where the second mapping splicing constituent units that are not matched with the first mapping splicing constituent unit are located region; step S460 includes: unfolding the anchor points in the overall data along the common coordinate axis to obtain the pixel coordinates of each anchor point in the overall unfolded image; The pixel value of the pixel point corresponding to the point and/or the pixel value of the pixel point corresponding to the anchor point in the second illumination light image determine the pixel value of the anchor point in the overall expanded image; for the anchor point located in the first non-public area point, according to the pixel value of the pixel point corresponding to the anchor point in the first illumination light image, determine the pixel value of the anchor point in the overall expanded image; for the anchor point located in the second non-common area, according to the The pixel value of the pixel corresponding to the anchor point determines the pixel value of the anchor point in the overall expanded image.

An anchor point is any point on the overall data. It can be understood that, in this embodiment, the overall data is an overall object model obtained by splicing the first object model and the second object model. The overall target model is a three-dimensional model, which may be composed of point clouds. Any point on the point cloud can be regarded as an anchor point. For example and not limitation, the coordinates of each anchor point in the three-dimensional space are determined, but the pixel value may be unknown. For the example in which the first data to be spliced and the second data to be spliced are three-dimensional target models, in step S460, during the process of unfolding the overall data along the common axis, the unfolding is performed in units of anchor points, so that each anchor Points are in one-to-one correspondence with each pixel in the illumination light image, thus, the pixel value of each pixel in the illumination light image can be One is mapped to the overall unfolded image. In addition, similar to the method for determining the pixel value of each pixel in the overall data adopted in the aforementioned mosaic expanded image solution, the common area and the non-common area can be divided to perform pixel value mapping. For an anchor point in a non-public area, the pixel value of the anchor point can be determined directly by using the corresponding pixel value of the illumination light image to which the non-public area belongs. As for the anchor points in the common area, you can select any illumination light image or combine the corresponding pixel values on two illumination light images to determine the pixel value of the anchor point, and then obtain the pixel values of each anchor point in the overall expanded image. Those skilled in the art can understand this method, and will not repeat it here.

An embodiment of the present application provides a handprint collection system. The system can use the blue light source and the green light source together to supplement light (or illuminate) for the image acquisition device when the image acquisition device collects the handprint image. Supplementing light with blue light and supplementing light with green light have their own advantages and disadvantages for handprint collection under different skin conditions. At the same time, using these two light sources to supplement light can expand the applicability of the handprint collection system to different skin conditions, thereby It is conducive to taking into account the collection needs of various groups of people.

According to one aspect of the present application, a handprint collection system is provided. You can refer back to FIG. 2 to understand the handprint collection system in this embodiment.

As shown in FIG. 2 , the handprint collection system 200 may include one or more image collection devices, an illumination system and a processing device 240 . The lighting system includes one or more blue light sources and one or more green light sources, and each light source is used to emit light toward the handprint collection area 300 . The blue light sources included in the lighting system form a blue light source group, and the green light sources included in the lighting system form a green light source group. The handprint collection area 300 is used to place part or all of the user's hand, such as the fingers 400 . One or more image acquisition devices are used to collect images of the handprint collection area 300 while the illumination system emits light to the handprint collection area 300 to obtain a handprint image. The processing device 240 is used for processing the handprint images collected by one or more image collection devices. It should be noted that the light source included in the lighting system described herein may be any type of light source not used for encoding, including but not limited to one or more of the following: point light source, line light source, surface light source, etc. The point light source may be, for example, a light strip composed of multiple point light sources. Exemplarily, at least part of the light sources in the lighting system and one or more image capture devices are located on the same side of the handprint collection area, for example, below the handprint collection area. The light source located on the same side of the handprint collection area as one or more image collection devices may be called the main light source. As an example and not limitation, in addition to the main light source, the lighting system may also include an auxiliary light source located near the handprint collection area, so as to illuminate the handprint from a position closer to the handprint. Compared with the main light source, the auxiliary light source is closer to the center of the handprint collection area. For example, the main light source can be arranged below the handprint collection area, and the center of gravity of the auxiliary light source (which can be understood as the respective center of gravity of each light source in the auxiliary light source) can be located or approximately on the same level as the center of the handprint collection area. At least part of the light sources in the auxiliary light sources can be located at predetermined positions on both sides of the handprint collection area (the first side of the two sides corresponds to the first predetermined position, and the second side corresponds to the second predetermined position), and/or at least part of the light sources It may be located at a predetermined position near the fingertip (which may be referred to as a third predetermined position). A light source located at a predetermined position on either side of the handprint collection area can emit light toward the side of the handprint collection area. A light source located near the fingertip location may emit light toward the fingertip location. The fingertip position refers to the expected position where the user's fingertip should fall when the user places the finger or palm on the handprint collection area. It is understandable that the position of the auxiliary light source is The goal is to illuminate the hand without affecting the imaging of the handprint acquisition system (for example, the auxiliary light source will not enter the imaging range of the image acquisition device). The auxiliary light sources may also contain one or more blue light sources and one or more green light sources.

Exemplarily, the handprint collection system of the embodiment of the present application can be non-contact, that is, the handprint collection area is a space, and the user can collect part or all of the hand in the handprint collection area, and there is no contact with the handprint collection area. The entity touched by the handprint. Of course, optionally, the handprint collection system can also be contact type, for example, a handprint collection system that realizes handprint collection by touching a finger with a screen running on a smart device.

FIG. 2 shows three image acquisition devices 212 , 214 and 216 . Furthermore, Fig. 2 also shows a lighting system comprising at least one blue light source and at least one green light source.

Exemplarily but not limitatively, the lighting system shown in FIG. 2 may include three light source sets corresponding to the three image acquisition devices, each light source set includes at least one blue light source and at least one green light source. Illumination is performed when the corresponding image acquisition device performs image acquisition.

Exemplarily, according to the difference in the relative positional relationship between the light sources in the light source set and the image acquisition device corresponding to the light source set, the light source set may be further divided into light source sub-sets. For example, the light sources in the light source set are distributed on both sides of the image acquisition device, the left light source is used as a light source sub-set, and the right light source is used as a light source sub-set. For example, in FIG. 2, a first set of light sources includes light source subsets 222 and 222', a second light source set includes light source subsets 224 and 224', and a third light source set includes light source subsets 226 and 226'.

It should be noted that the structure of the handprint collection system 200 shown in FIG. 2 is only an example and not a limitation to the application, and the handprint collection system 200 according to the embodiment of the application is not limited to the situation shown in FIG. 2 . For example, the number of image capture devices may be more or less than three. In addition, the number of light source subsets included in the light source set around each image acquisition device shown in FIG. 2 and the positions of the light source subsets can also be changed as required. For example, any set of light sources may also include a single subset of light sources (eg, the first set of light sources may only include subset 222 of light sources) or more than two subsets of light sources.

Exemplarily, the lighting system includes at least one light source pair, and each light source pair includes a blue light source and a green light source. This example is a preferable way to arrange the light sources. This way makes each paired blue light source and green light source as close as possible, so that the acquisition conditions of the blue channel and the green channel of the image are as close as possible to the same. It helps to improve the accuracy of subsequent handprint images obtained based on the two channels.

Exemplarily, in an embodiment where the lighting system includes one or more light source sets corresponding to one or more image acquisition devices, each light source set may include at least one light source pair, and each light source pair includes a blue light source and a green light source. In this way, it is convenient to make the imaging conditions of the blue channel and the green channel corresponding to each image acquisition device as close as possible to the same when acquiring images. For example, any subset of light sources in FIG. 2 may include a pair of light sources.

The lighting system includes at least one blue light source and at least one green light source. On this basis, the number of light sources in the lighting system can be set to any suitable number as required. If classified according to color, the light sources included in the lighting system can be divided into a blue light source group and a green light source group. The number of blue light sources included in the blue light source group and the number of green light sources included in the green light source group can be arbitrary, and they can be consistent or inconsistent. Preferably, the number of blue light sources included in the blue light source group is consistent with the number of green light sources included in the green light source group.

Exemplarily, the handprint collection system 200 may further include a casing, and at least part of the light sources in the one or more image collection devices and the lighting system may be disposed inside the casing. Exemplarily, a window may be provided on the casing, and a light-transmitting plate may be provided on the window. The handprint collection area can be located above the light-transmitting plate, and the user puts a finger in the handprint collection area, and the lighting system below can shine light on the finger through the light-transmitting plate. The image acquisition device below the handprint acquisition area can also acquire fingerprint images of fingers through the light-transmitting plate. Of course, the embodiment in which the handprint collection area is located above the light-transmitting plate is only an example, the light-transmitting plate is optional, and the position of the handprint collection area relative to at least part of the light sources in the image collection device and lighting system can also be changed, as long as The handprint collection area only needs to be located within the imaging range of the image collection device and within the illumination range of the lighting system. In the description herein, the imaging range of any image acquisition device refers to the range covered by a cone with the optical center of the lens of the image acquisition device as the vertex and the viewing angle of the image acquisition device as the cone angle. It should be noted that the light-transmitting plate can be installed in the casing at a set height, so as to prevent the virtual image formed by the illumination source through the light-transmitting plate from entering the imaging range of the image acquisition device. It can be understood that the lighting system may also include a light source located near the handprint collection area, that is to say, the lighting system may also include an auxiliary light source located above the light-transmitting plate, so as to illuminate the hand at a position closer to the hand. The auxiliary light source can also include both blue light source and green light source. The auxiliary light source can include one or more of point light source, line light source, and surface light source, and can be located near the hand in the handprint collection area. For example, the auxiliary light source includes light strips composed of point light sources on both sides of the handprint collection area. and a point light near the fingertip location.

After the user places the finger or palm in the handprint collection area, the image collection device can collect the user's fingerprint or palmprint image. The handprint collection system can be used to collect only one type of hand texture, or it can be used to collect multiple types of hand textures. For example, the handprint collection system can only be used to collect fingerprints in the fingertip area (including fingerprints on the front and side of the finger pad), and can also be used to collect fingerprints in the fingertip area and palm prints at the same time, and can also be used to collect Fingerprints in the fingertip area, lines on the knuckles of the fingers except the knuckles where the fingertips are located, and palm prints on the palm. Multiple types of hand textures can share the same handprint collection area, or use different handprint collection areas, and the handprint collection areas of different types of hand textures can partially or completely overlap.

The number of image capture devices included in the handprint capture system 200 can be set to any appropriate number as required. For example, the number of image acquisition devices may be 1, 2, 3 or 6 and so on.

In one embodiment, the handprint collection system 200 may include a single image collection device. this In this case, optionally, the image of the handprint at a single angle can be collected by an image collection device. Of course, optionally, in this case, it is also possible to collect handprint images from multiple angles by rotating and/or moving the image collection device. When the collected handprint is a fingerprint, the implementation scheme of collecting the handprint image at a single angle is more suitable for simulating the fingerprint image obtained by flat printing.

In another embodiment, the handprint collection system 200 may include multiple image collection devices. In this case, optionally, multiple image capture devices may be used to capture handprint images from multiple angles. That is, the optical axes of any two image capture devices among the plurality of image capture devices may form a preset angle with each other, and the preset angle is not equal to 0, so as to collect handprint images at different angles. When the collected handprint is a fingerprint, the angle/height of multiple image acquisition devices can be adjusted so that not only the frontal fingerprint image of the finger pad can be collected, but also the side fingerprint image of the finger pad can be collected, and the handprint image from multiple angles can be collected The implementation scheme of is more suitable for simulating the fingerprint image obtained by rolling stamping. It can be understood that multiple image acquisition devices may be arranged such that their optical axes are approximately perpendicular to the handprint area to be acquired. For example, the optical axis of the image acquisition device used to collect fingerprint images on the front of the finger pad is perpendicular to the front area of the finger pad, and the optical axis of the image acquisition device used to collect fingerprint images on the side of the finger pad is perpendicular to the side area of the finger pad. When the handprint acquisition system is used to acquire multiple types of hand textures, the plurality of image acquisition devices may include an image acquisition device for acquiring a first type of hand texture and an image acquisition device for acquiring a second type of hand texture. The field of view, focal length, placement angle, and position of the image acquisition device for acquiring the first type of hand texture and the image acquisition device for acquiring the second type of hand texture may be different. It can be understood that, compared with the image collection device used for collecting fingerprints, the image collection device for collecting palmprints has a larger field of view. The field of view of an image acquisition device used to collect fingerprints or palmprints may refer to the field of view of a single image acquisition device, or may refer to the field of view of a combination of multiple image acquisition devices. For the same image acquisition device, it can be used to acquire both the first type of hand texture and the second type of hand texture.

It can be understood that the image acquisition device may also include a preview image acquisition device dedicated to preview, and the preview image acquisition device may have a larger field of view than the image acquisition device used for shooting, for example, the field of view of a single preview image acquisition device covers the entire palm The common field of view of multiple image acquisition devices used for shooting covers the entire palmprint. Multiple image acquisition devices can capture multiple palmprint images of target parts such as fingers or palms of the user in one-to-one correspondence, and multiple palmprint images can correspond to different parts of the target site. For example, there are three image acquisition devices, the handprint image collected by the first image acquisition device can mainly correspond to the left part of the finger or palm (i.e. the side area, specifically the side area on the left side), and the handprint image collected by the second image acquisition device The palmprint image captured by the image capture device may mainly correspond to the middle part of the finger or palm (i.e. the frontal area), while the palmprint image captured by the third image capture device may mainly correspond to the right side part of the finger or palm (i.e. side area, specifically the side area on the right side).

The optical axis of the image acquisition device is set towards the side where the handprint of the target part is located, for example, the pad of the finger or the palm of the palm. For example, if it is stipulated that the pads of the fingers of the user face the ground plane when collecting the fingerprints of the user, the optical axis of the image collection device can be set vertically upward or obliquely upward so as to collect the fingerprint of the user's finger.

When there are multiple image capture devices, the heights of the centers of gravity of the multiple image capture devices may be set to be consistent or inconsistent. For example, in the case of collecting palm prints, the heights of the centers of gravity of multiple image capturing devices may be consistent. In the case of collecting handprints, the centers of gravity of multiple image capturing devices may lie on a circular arc.

The processing device 240 is communicatively connected to one or more image acquisition devices, which connection may be wired or wireless. The processing device 240 may receive handprint images collected by one or more image collection devices, and process the handprint images. The processing device 240 can perform any necessary processing operations on the handprint image, including but not limited to one or more of the following: obtaining depth information from the handprint image; performing splicing processing on the handprint images collected by multiple image acquisition devices, etc. wait.

By way of example and not limitation, the processing device 240 can also be communicatively connected with the lighting system, and the connection can be wired or wireless. In this case, the processing device 240 may control each light source in the lighting system to emit light according to various lighting schemes described herein. Of course, alternatively, other control devices different from the processing device 240 may also be used to control each light source in the lighting system to emit light. The above-mentioned control device may be a control device in the handprint collection system, or an external control device independent of the handprint collection system.

Different people's skin has different reflection, absorption and transmittance of light, and the same person's hand (finger or palm as an example) has different reflection, absorption and transmittance of light of different wavelengths. In addition, the reflection, absorption and transmission of light are also different when the fingers or palms of the same person are dry, wet or sweaty.

The inventors found that when collecting handprint images, if only green light is used to supplement the light, since the green light is strongly absorbed by the fingers or palms, on the one hand, the light reflected from the fingers or palms to the image acquisition device becomes weaker, which reduces the contrast of the image. On the other hand, the absorption of light by fingers or palms will produce fluorescence and phosphorescence, which will blur the collected images; if only blue light is used to fill the light, due to the weak absorption of blue light by fingers or palms, the fingers or palms will reflect to the image acquisition device The light is strong, and it is easy to overexpose for people with light skin. In addition, if the hands are too dry, there is dead skin or liquid such as sweat, there will be strong reflections, which will easily cause poor contrast. However, both blue light and green light have their own advantages. For example, the contrast of the ridges and valleys of fingerprints or palm prints will be improved under the blue light fill light, which is suitable for people with light fingerprints; while the overdry or wet fingers can be photographed more clearly under the green light fill light.

Therefore, the inventor chooses to supplement the light with blue and green light sources at the same time, so that the two lights can complement each other. As long as a clear image can be collected under one color of light, high-quality images can be obtained relatively easily after processing. Handprint image. Therefore, this scheme of supplementing light with blue and green light sources when collecting handprints can expand the applicability of the handprint collection system to different skin conditions, thereby helping to meet the collection needs of various groups of people.

In order to avoid the mutual influence of blue and green light, the wavelength discrimination of blue and green light sources can be set as large as possible, that is, blue light sources with shorter wavelengths (purple) and green light sources with longer wavelengths (yellow) can be selected as much as possible. By way of example and not limitation, the blue light described herein may be light with a wavelength of less than 430nm, such as light at 405nm, and the green light may be light with a wavelength greater than 540nm, such as 560nm light.

In a specific embodiment, the lighting system may include one or more light source sets, each light source set includes at least one blue light source and at least one green light source, one or more light source sets are connected with one or more image acquisition devices one by one correspond.

In this embodiment, the light source set corresponding to any image acquisition device is responsible for supplementing light for the image acquisition device, and each time the image acquisition device captures an image, only the light source set corresponding to it can emit light, and other light source sets do not emit light . Assigning a corresponding supplementary light source to each image acquisition device facilitates the control and management of the image acquisition device and the light source, and facilitates the avoidance of mutual interference of different light sources when acquiring images.

As an example and not limitation, for any light source set in one or more light source sets, the direction of the light emitted by the light source in the light source set may be the same as the handprint collected when the light source in the light source set is turned on. The area is roughly vertical. In this way, the light is approximately perpendicular to the handprint area, which is beneficial to ensure that the valleys and ridges of the handprint can be more accurately presented in the handprint image. When multiple image acquisition devices can be arranged so that their optical axes are roughly perpendicular to the handprint area to be collected, the direction in which the light sources in each light source set emit light is roughly parallel to the direction of the optical axis of the corresponding collection device, which can be achieved by This is achieved by arranging the light sources in the light source set and their corresponding image acquisition devices close enough.

With continued reference to FIG. 2 , optical axes 2122 , 2142 , and 2162 of image capture devices 212 , 214 , and 216 , respectively, are shown. It can be seen from FIG. 2 that the direction of the light emitted by the light sources in each light source set is parallel to the direction of the optical axis of the corresponding collection device. It should be noted that the parallel described herein does not necessarily require parallel objects to be absolutely parallel to each other, but may be approximately parallel. For example, parallel may mean that the angle between a parallel object (in this embodiment, the direction of the light emitted by the light source and the direction of the optical axis of the corresponding image acquisition device or any two light rays emitted by the same parallel light source) is smaller than a preset threshold . The threshold can be set as small as possible, for example, set to 10° or the like.

According to an embodiment of the present application, the lighting system includes at least one light source pair, each light source pair includes a blue light source and a green light source, and the distance between the blue light source and the green light source in the same light source pair is smaller than a preset distance threshold.

During image acquisition, the blue light source and the green light source in the same light source pair can be turned on or off at the same time, which is more conducive to ensuring that the imaging conditions of the blue channel and the green channel are closer to the same. The same imaging conditions of the blue channel and the green channel help to improve the fusion effect of the blue channel image and the green channel image in the subsequent image fusion.

However, this embodiment is only an example and not a limitation to the present application. The light sources in the lighting system may all be unpaired, or the lighting system may include light source pairs and unpaired light sources.

Each light source set may include at least one light source pair. Each pair of light sources may include a blue light source and a green light source. The blue light source and the green light source in each light source pair can be as close as possible to further ensure that the imaging conditions of the blue channel and the green channel are close to the same. For example, referring back to FIG. 2, any subset of light sources (eg, 222, 222', 224, 224', 226, 226') may include at least one pair of light sources, so that the blue light source and the green light source in the light source subset can be conveniently arranged closer.

The preset distance threshold may be set to any appropriate value as required, and this application is not limited thereto.

According to an embodiment of the present application, the illumination system is located outside the imaging range of one or more image acquisition devices. In embodiments where the illumination system includes one or more sets of light sources, the set of light sources corresponding to any image capture device may be located outside the imaging range of that image capture device.

The light source can be arranged at any suitable position around the image acquisition device, as long as it does not hinder the imaging of the image acquisition device. The set of light sources corresponding to any image capture device is set outside the imaging range of the image capture device, which can effectively prevent the light source from hindering the image capture device from collecting handprint images.

According to an embodiment of the present application, one or more image acquisition devices include a first image acquisition device, a second image acquisition device and a third image acquisition device, wherein the optical axis of the first image acquisition device is aligned with the direction of the handprint acquisition area Or the planes of multiple image acquisition devices are vertical, the optical axis of the second image acquisition device forms a first preset angle with the optical axis of the first image acquisition device, the optical axis of the third image acquisition device and the first image acquisition device The optical axis forms a second preset angle.

The plane of the handprint collection area facing one or more image collection devices may be, for example, a plane parallel to the plane where the above-mentioned shading plate is located. When it is stipulated that the pads of the fingers or the palm of the palm of the user must face the ground plane when the user collects the fingerprints, the plane of the palmprint collection area facing one or more image capture devices can be parallel to the ground plane. At this time, the optical axis of the first image acquisition device may be vertically upward, and the optical axes of the second image acquisition device and the third image acquisition device may be obliquely upward.

Any one of the first preset included angle and the second preset included angle may be set to any appropriate value as required, which is not limited in this application. The first preset included angle and the second preset included angle may be the same or different. Exemplarily, the first preset included angle may be in the range of 0 to 45 degrees, and the second preset included angle may also be in the range of 0 to 45 degrees.

According to the embodiment of the present application, the optical axes of the first image acquisition device, the second image acquisition device and the third image acquisition device are located in the same plane.

The arrangement of the optical axes of the three image acquisition devices in the same plane and arranged non-parallel to each other is simple, and also helps to more conveniently acquire images of multiple different parts of target parts such as fingers or palms.

According to the embodiment of the present application, the handprint collection system may further include: a structured light projector for emitting structured light to the handprint collection area; Handprint images are processed: the grayscale images of the channels corresponding to the structured light are extracted from the handprint images collected by one or more image acquisition devices, and the depth information of the handprints is determined based on the extracted structured light images.

Figure 14 shows a handprint collection system 200 and related fingers according to one embodiment of the present application 400 and a schematic diagram of the handprint collection area 300. FIG. 14 shows that the handprint collection system 200 also includes a structured light projector 100 . FIG. 2 can be understood as a front view of the handprint collection system 200 , and FIG. 14 can be understood as a left side view of the handprint collection system 200 . It should be noted that FIG. 14 only shows a single image acquisition device 214 as an example for description, and those skilled in the art can understand the positions of other image acquisition devices in FIG. 14 .

The structure of the handprint collection system 200 shown in FIG. 14 is only an example and not a limitation to the application, and the handprint collection system 200 according to the embodiment of the application is not limited to the situation shown in FIG. 14 . For example, FIG. 14 shows that the structured light projector 100 is located on the side closer to the root of the finger with a specific cross-section 302 as the boundary. The opposite part) is farther horizontally from the root of the finger, wherein the specific cross-section 302 is the cross-section of the cross-section of the finger 400 that overlaps with the central axis of the handprint collection area 300 . The arrangement of the above-mentioned structured light projector 100 is only an example, and it can also be located on the side farther away from the root of the finger bounded by the specific cross section 302, that is, the head of the structured light projector 100 can also be farther away from the root of the finger than the tail. The horizontal distance is closer. In fact, the structured light projector 100 can be arranged at any suitable position as required, as long as it does not hinder the imaging of the image acquisition device and can illuminate the handprint acquisition area.

By way of example and not limitation, the height of the center of gravity of the structured light projector 100 and the height of the integrated center of gravity or the lowest center of gravity of one or more image acquisition devices may be set to be substantially the same. In this way, it is convenient to arrange the components in the handprint collection system more compactly, and it is beneficial to reduce the volume of the device. By way of example and not limitation, the height of the center of gravity of the structured light projector 100 can be lower than the height of the integrated center of gravity or the lowest center of gravity of one or more image acquisition devices, so that the light projection of the structured light projector 100 can be ensured. The area is large enough to cover the acquisition area corresponding to the target site as effectively as possible.

In the case of collecting fingerprints, the target site is relatively small, and the height of the center of gravity of the structured light projector 100 can be set to be substantially the same as the height of the integrated center of gravity or the lowest center of gravity of one or more image acquisition devices. In the case of collecting palm prints, the target site is relatively large, and the height of the center of gravity of the structured light projector 100 may be lower than the height of the combined center of gravity or the lowest center of gravity of one or more image capture devices.

The grayscale image of the channel corresponding to the structured light extracted from any handprint image is the structured light image corresponding to the handprint image. It can be understood that the structured light image includes depth information corresponding to each pixel in the original handprint image. Therefore, through the structured light projector, the depth information of the palm lines of target parts such as fingers or palms can be further obtained. Obtain the depth information of the handprint, which is convenient for subsequent simulation of the handprint image. At present, many occasions may need to be applied to fingerprint images obtained by stamping. Therefore, the scheme of adding structured light can correctly expand the acquired handprint image into a two-dimensional image, so that it can be compared with the fingerprint image obtained by stamping in the subsequent comparison process. Compatible with fingerprint images, expanding the application scenarios of the handprint collection system.

As an example and not a limitation, since two colors of blue and green are used for supplementary light, in order to reduce the interference to the blue and green channel, the structured light can try to choose a color that is quite different from the blue and green, for example Such as red and so on. Exemplarily, the structured light emitted by the structured light projector may be red light with a wavelength of 660nm. Illumination light and structured light respectively choose blue, green, and red three colors, which is conducive to the subsequent convenient extraction of images of different channels. Of course, red light may not be used for structured light, and image processing can be used to remove the influence of structured light in the blue-green channel.

According to the embodiment of the present application, the direction of the light emitted by the structured light projector and the plane where the optical axes of the one or more image acquisition devices are located form a third preset angle.

The third preset angle can be set to any suitable value as required, and this application is not limited thereto. The third preset angle may be different from the above-mentioned first preset angle and the second preset angle, or may be the same as the first preset angle and/or the second preset angle.

It is assumed that the optical axes of all the image acquisition devices in FIG. 14 are in the same plane, and this plane appears as a straight line parallel to the optical axis 2142 in the left view shown in FIG. 14 . It can be seen from FIG. 14 that the direction of light emitted by the structured light projector 100 forms a certain angle with the plane where the optical axis of the image acquisition device is located. This method of arranging the structured light projector on the side of the image acquisition device facilitates the structured light projector to avoid the imaging range of the image acquisition device, so as to avoid hindering the imaging of the image acquisition device. In addition, this method of arranging the structured light projector on the side of the image acquisition device also facilitates the arrangement of the structured light projector at a height similar to that of the image acquisition device, which is conducive to achieving a more compact layout inside the handprint acquisition system.

According to the embodiment of the present application, the handprint collection system 200 may also include a casing, one or more image collection devices and lighting systems are arranged in the casing, a window is set on the casing facing the handprint collection area, and a window is set on the window. There are transparent panels.

The above describes the embodiment in which the image acquisition device and the lighting system are arranged in the casing, and a light-transmitting plate is arranged on the casing, and details will not be repeated here. The light-transmitting plate can be realized by using any suitable material, including but not limited to glass and the like. Exemplarily, the glass may be glass with an anti-reflection coating. The light-transmitting board can play the role of waterproof and dustproof, help protect the main structure of the handprint collection system, and prolong the service life of the equipment.

According to the embodiment of the present application, the handprint collection system 200 may also include a limiting part, which defines a handprint collection area, and an inlet is provided at the proximal end of the limiting part to allow part or all of the user's hand to pass through the inlet. Enter the handprint collection area; the far end of the limiting component is provided with a stopper, and the stopper is located at a predetermined position in the handprint collection area along the extending direction, so as to limit part or all of the user's hand from extending into the handprint collection area location within.

The proximal end of the limiter can be understood as the end of the limiter that is closer to the front of the user when the user normally uses the handprint collection system, and the far end of the limiter can be understood as the limiter when the user normally uses the handprint collection system on the end that is farther from the front of the user.

Fig. 15 shows a schematic diagram of a part of the handprint collection system according to one embodiment of the present application. In FIG. 15 , the limiting component 250 is shown, and the inlet 252 and the stop portion 254 of the limiting component 250 are shown. The area between the entrance 252 and the stopper 254 of the limiting component 250 can be set as the handprint collection area 300 . Target parts such as user's finger or palm can reach into hand through inlet 252 In the pattern collection area 300. The stopper 254 can limit the limit position that the user's finger or palm can reach, and prevent the user from extending the target part such as finger or palm too far. Through the limit part 250, it is convenient to guide the user to place the target part such as the finger or the palm in the correct position, so as to ensure that the part on the target part that is more suitable for detection (for example, the first knuckle of the finger closest to the fingertip) can Fall into the handprint collection area.

According to the embodiment of the present application, the handprint collection system 200 may further include a light-shielding component, and the light-shielding component and at least part of the light sources in the lighting system are respectively located on opposite sides of the handprint collection area.

Referring to FIG. 15 , it shows a shading member 260 , which is located on opposite sides of the handprint collection area 300 with at least part of the light sources in the lighting system. Wherein, the shading component is located above the handprint collection area, and at least part of the light sources in the lighting system are located below the handprint collection area. The shading component 260 can prevent the light emitted by the light source from reaching the outside to affect the user experience, and at the same time prevent the stray light from the outside from shining into the handprint collection area to interfere with the collection of the handprints.

In the case where the handprint collection system includes a light-shielding component and a light-transmitting plate, the handprint collection area may be at least a part of the area between the light-shielding component and the light-transmitting plate.

Fig. 16 shows a schematic diagram of a partial structure of a handprint collection system 200 and related fingers and handprint collection areas according to an embodiment of the present application. It should be noted that FIG. 16 is a schematic diagram and does not represent the actual shape or size of each component. Referring to FIG. 16 , it shows a light-shielding member 260 and a light-transmitting plate 270 , and a handprint collection area 300 is disposed therebetween. In addition, FIG. 16 also shows two groups of auxiliary light sources 2281 and 2282 on both sides of the handprint collection area 300 and an auxiliary light source 2283 near the fingertip position of the handprint collection area 300 . In the example shown in Figure 16, two groups of auxiliary light sources 2281 and 2282 on both sides of the handprint collection area 300 are respectively a light strip, and the auxiliary light source 2283 near the fingertip position of the handprint collection area 300 is a point light source, but this is only is an example, the auxiliary light source may have other implementation forms. In addition, it should be noted that, in FIG. 16 , the finger 400 is represented by a dotted line for convenience of distinction.

According to the embodiment of the present application, in the case that the handprint collection system further includes the above-mentioned limiting part 250, the handprint collection system may also include a light-shielding part 260, and the light-shielding part 260 and the lighting system are respectively located on opposite sides of the handprint collection area. ; The limiting component 250 also includes a connecting portion connecting the inlet and the stop portion, and the upper surface of the connecting portion is attached to the lower surface of the light-shielding portion.

Referring to FIG. 15 , a connection portion 256 connecting the inlet 252 and the stop portion 254 in the limiting member 250 is shown. The upper surface of the connecting part 256 can be attached to the lower surface of the light shielding part 260, which can better prevent the light emitted by the light source from escaping to the outside and better prevent outside stray light from irradiating the handprint collection area.

According to another aspect of the present application, a handprint collection method implemented by the above-mentioned handprint collection system 200 is provided. The handprint collection method includes: when the blue light source group emits light with a blue light-emitting scheme, and the green light source group emits light with a green light-emitting scheme, one or more image collection devices are used to collect images of the handprint collection area to obtain one or more The handprint images collected by each image acquisition device, so that in the handprint images collected by each image acquisition device, at least one handprint image is in the case where there is a blue light source with a luminous intensity other than 0 in the blue light source group Collected, at least one handprint For example, it is collected when there is a green light source with a luminous intensity other than 0 in the green light source group; wherein, the blue light source group includes one or more blue light sources, the green light source group includes one or more green light sources, and the blue light source group includes one or more green light sources. The luminous scheme is used to characterize the luminous intensity of each blue light source, and the green luminous scheme is used to characterize the luminous intensity of each green light source; when collecting, the luminous intensity of at least one light source is not 0.

The structure and arrangement of the lighting system and the image acquisition device can be understood in combination with the above description, and will not be repeated here.

As mentioned above, the lighting system can be divided into two groups according to the colors of the light sources, namely, the blue light source group and the green light source group. The blue light source group may include all blue light sources in the lighting system, and the green light source group may include all green light sources in the lighting system.

For each image acquisition device in the handprint acquisition system, it is only necessary to ensure that at least one of the handprint images collected by it is the case where there is a blue light source with a luminous intensity other than 0 in the blue light source group At least one handprint image is collected when there is a green light source with a luminous intensity not equal to 0 in the green light source group. The time sequence of image collection by all image acquisition devices and the light emission time sequence of all light sources in the lighting system are the same. Can be set as required.

Among the "at least one handprint image" collected when the luminous intensity of the blue light source is not 0 and the "at least one handprint image" collected when the luminous intensity of the green light source is not 0, there may be one or multiple handprint images are the same. For example, for any image acquisition device X, it can acquire two handprint images in total. The first one is when the luminous intensity of at least one blue light source is not 0 and the luminous intensity of all green light sources is 0. The second one is collected when the luminous intensity of at least one green light source is not 0 and the luminous intensity of all blue light sources is 0. For another example, the image acquisition device X may only acquire one handprint image, and the handprint image may be acquired when the luminous intensity of at least one blue light source is not zero and the luminous intensity of at least one green light source is not zero. . For another example, the image acquisition device X can also acquire three handprint images, the first one is acquired when the luminous intensity of at least one blue light source is not 0 and the luminous intensity of all green light sources is 0, the second The second is collected when the luminous intensity of at least one blue light source is not 0 and the luminous intensity of at least one green light source is not 0, and the third is collected when the luminous intensity of at least one green light source is not 0 and all Collected when the luminous intensity of the blue light source is all 0. The above collection schemes of the image collection device X are all feasible, and of course, other numbers of images can be collected and other lighting methods can be used, which will not be repeated here.

For each acquisition of one or more image acquisition devices, the acquisition corresponds to a blue light-emitting scheme and a green light-emitting scheme, and the handprint image obtained in this acquisition is also corresponding to the blue light-emitting scheme. Corresponding to this green lighting scheme. Each blue light emission scheme is used to characterize the luminous intensity of each light source in the blue light source group, and each green light emission scheme is used to characterize the luminous intensity of each light source in the green light source group.

The following takes the blue light source group as an example to illustrate the light source in this paper according to the light emission scheme and the image acquisition. The relationship between the green light source group and the blue light source group is similar, so this article will not go into details. Assuming that the blue light source group includes three blue light sources B1, B2, and B3, and the luminous intensity of each light source is only divided into two levels, one is when the intensity reaches the maximum, and the other is when the intensity is 0, then theoretically there are (ie There are up to) eight blue glow schemes. Assuming that the maximum luminous intensity of each light source is represented by the number "1", and the intensity of 0 is represented by the number "0", so the above eight blue light-emitting schemes can be expressed as 000, 001, 010, 100, 011, 101, 110 , 111. For the image acquisition device, some or all of the eight blue light-emitting schemes can be used to sequentially emit light, and a corresponding handprint image can be collected under each light-emitting scheme. For example, for the image acquisition device, a total of six blue light-emitting schemes are used to emit light in any order. When the blue light source group emits light according to any blue light-emitting scheme each time, at least part of the images in one or more image acquisition devices The collecting device collects the corresponding handprint image. It should be noted that the same blue light-emitting scheme can appear one or more times. For example, any blue light-emitting scheme P among the above-mentioned six blue light-emitting schemes appears three times, and other blue light-emitting schemes appear once. In the end, eight images, and the emission schemes corresponding to the three images are blue emission scheme P.

It should be noted that, for the image acquisition device, no matter which blue light emitting scheme or green light emitting scheme is used to emit light, the luminous intensity of at least one light source is not 0 during each acquisition.

Although the luminous intensity of each light source is only divided into two levels for description above, it can be understood that the luminous intensity of each light source can have more levels, and the number of luminous intensity levels of any two different light sources They can be the same or different, and these can be set arbitrarily. In the case that the luminous intensity of the blue/green light source has more gears, the maximum possible number of blue/green luminous schemes can also be increased, but the blue/green luminous schemes finally used to supplement light for any image acquisition device can be It can be selected as required, and it can include all possible blue/green light emitting schemes, or only a part of all possible blue/green light emitting schemes.

According to the handprint collection method of the embodiment of the present application, for each image collection device, at least one of the collected handprint images is collected when the luminous intensity of the blue light source is not 0, and at least one is Collected when the luminous intensity of the green light source is not 0. Therefore, in this method, when each image capture device captures an image, the blue light source and the green light source can be used together to supplement light for the image capture device. This can expand the applicability of the handprint collection method to different skin states, thereby helping to take into account the collection needs of various groups of people.

According to an embodiment of the present application, the above collection includes multiple times, and at least two of the multiple collections correspond to different blue light emission schemes, and/or, at least two of the multiple collections correspond to different green light emission schemes.

For one or more image acquisition devices, the operation of acquiring handprint images can be performed multiple times. The image acquisition devices performing any two different acquisition operations may be the same or different. When controlling the lighting system to emit light, the lighting scheme corresponding to at least two of the above-mentioned multiple collections can be changed. The changed lighting scheme can be a blue lighting scheme, a green lighting scheme, or a blue lighting scheme. Both the scheme and the green glow scheme change. in blue When both the light-emitting scheme and the green light-emitting scheme are changed, the at least two acquisitions corresponding to the change of the blue light-emitting scheme and the at least two acquisitions corresponding to the change of the green light-emitting scheme may be completely different, completely the same, or only Part of the collection is the same.

The at least two collections with different blue light emitting schemes may be continuous at least two collections, or may be partially or completely discontinuous at least two collections. Similarly, the at least two acquisitions with different green light emission schemes may be at least two acquisitions that are continuous, or at least two acquisitions that are partly or completely discontinuous.

The collection situation faced by the handprint collection system is more complex and will change at any time. For example, different people have different skin conditions, and different people have different positions to place the target parts. Also subject to change. According to different acquisition conditions, the luminescence conditions required to obtain a better-quality handprint image may also be different. If the luminescence scheme is stable during the process of collecting handprint images by the image acquisition device, and only single-quality handprint images can be collected, if the current luminescence scheme is not suitable for the current user's collection situation, it may cause the final obtained handprint image The texture collection effect is not ideal. Therefore, in the process of collecting the handprint image by the image acquisition device, the light-emitting scheme is changed to a certain extent, so that the quality of the collected handprint image can also be changed to a certain extent, which can make up for the limitation of a single handprint image to a certain extent. , which helps to obtain a better handprint collection effect.

According to the embodiment of the present application, the above collection includes multiple times, and the blue light emission schemes corresponding to every two collections in the multiple collections are different, and/or, the green light emission schemes corresponding to every two collections in the multiple collections are different.

The above has described the embodiment in which the lighting scheme corresponding to every two acquisitions changes, and the implementation manner of this embodiment can be understood in conjunction with the above description, and details are not repeated here. The luminescence scheme corresponding to every two acquisitions changes, and the luminescence scheme in this way is varied, which helps to better improve the acquisition effect.

According to the embodiment of the present application, the blue light emission scheme or the green light emission scheme corresponding to at least two acquisitions are the same.

In at least two acquisitions, the blue light-emitting scheme may be kept constant and only the green light-emitting scheme may be changed, or the green light-emitting scheme may be kept constant and only the blue light-emitting scheme be changed. Adopting this embodiment, generally speaking, the change span of the lighting scheme is small, and it is convenient to align the handprint images collected at different times based on the constant lighting scheme of the same color.

According to the embodiment of the present application, there are no two acquisitions corresponding to the same blue light emission scheme and the same green light emission scheme.

For the two collections, if the blue light-emitting scheme and the green light-emitting scheme are exactly the same, then two handprint images with basically the same quality will be collected, which is of little significance. For the image acquisition device, all its acquisition operations correspond to different blue light-emitting schemes and/or green light-emitting schemes, so that during the entire acquisition process of the image acquisition device, as many light-emitting schemes as possible can be changed to obtain more Handprint images of different qualities. This helps to improve the The probability of occurrence of the lighting scheme in the set situation helps to improve the probability of obtaining high-quality handprint images, and then helps to improve the final handprint collection effect.

According to the embodiment of the present application, when the blue light source group emits light in a blue light-emitting scheme, and the green light source group emits light in a green light-emitting scheme, collecting images of the handprint collection area by one or more image acquisition devices includes: at the first moment, When the blue light source group emits light with the first blue light-emitting scheme, and the green light source group emits light with the first green light-emitting scheme, at least part of the image capture devices in the one or more image capture devices are used to collect images of the handprint collection area; At the second moment, when the blue light source group emits light with the second blue light-emitting scheme, and the green light source group emits light with the second green light-emitting scheme, the handprint collection area is collected by at least part of the image collection devices in one or more image collection devices , wherein the first blue light-emitting scheme and the second blue light-emitting scheme are selected from the set of blue light-emitting schemes, the first green light-emitting scheme and the second green light-emitting scheme are selected from the set of green light-emitting schemes; in the set of blue light-emitting schemes Including w kinds of blue lighting schemes, where the maximum value of w is 2 ^k1 , in the 2 ^k1 blue lighting schemes, the luminous intensity of any blue light source is divided into two levels of 0 and non-zero; the set of green lighting schemes includes p kinds of green lighting schemes, where the maximum value of p is 2 ^k2 , in the 2 ^k2 kinds of green lighting schemes, the luminous intensity of any green light source is divided into two levels of 0 and non-0; k1 is the blue light source in the lighting system The total number, k2 is the total number of green light sources in the lighting system; w is an integer greater than or equal to 2 and/or p is an integer greater than or equal to 2; the first blue light-emitting scheme is different from the second blue light-emitting scheme, And/or, the first green lighting scheme is different from the second green lighting scheme.

In this embodiment, the luminous intensity of each light source in the lighting system is only divided into two levels, one is that the luminous intensity is not zero, and the other is that the luminous intensity is zero. Therefore, the set of blue light emitting schemes may include at most 2 ^k1 blue light emitting schemes. Similarly, the set of green lighting schemes may include at most 2 ^k2 green lighting schemes.

For any light source, when it is in a gear whose luminous intensity is not 0, the value of its luminous intensity can be any value, such as the maximum luminous intensity of the light source, or half of the maximum luminous intensity of the light source, and so on.

As mentioned above, during the process of image acquisition by the image acquisition device, the lighting scheme may change. In this embodiment, the first moment and the second moment may emit light according to different blue light emitting schemes and/or different green light emitting schemes respectively. The image acquisition devices that acquire the handprint images at the first moment and the second moment may be completely the same, completely different or partially the same.

This embodiment can be understood in combination with the above description about the change of the lighting scheme, and details are not repeated here.

According to an embodiment of the present application, the first blue light emitting scheme is the same as the second blue light emitting scheme, or the first green light emitting scheme is the same as the second green light emitting scheme.

Assume below that the blue light source group includes three blue light sources B1, B2, and B3, and the green light source group includes three green light sources G1, G2, and G3, and assume that the luminous intensity of each light source is not 0 and represented by the number "1", An intensity of 0 is represented by the number "0". At the first moment, the first blue light-emitting scheme adopted may be 100, and the first green light-emitting scheme may be 111; and at the second moment, the second blue light-emitting scheme adopted may still be 100, and the second green light-emitting scheme may be 100. Can be changed to 011. in this In the embodiment, if the blue light-emitting scheme changes, the green light-emitting scheme remains unchanged; otherwise, if the green light-emitting scheme changes, the blue light-emitting scheme remains unchanged.

The advantages of the embodiment in which the blue light emitting scheme and/or the green light emitting scheme are unchanged in at least two acquisitions have been described above, and will not be repeated here.

According to an embodiment of the present application, when the blue light source group emits light in a blue light-emitting scheme and the green light source group emits light in a green light-emitting scheme, collecting images of the handprint collection area by one or more image acquisition devices further includes: at the third moment , when the blue light source group emits light with the third blue light-emitting scheme, and the green light source group emits light with the third green light-emitting scheme, at least part of the image capture devices in the one or more image capture devices are used to capture images of the handprint collection area, Wherein, the third blue light-emitting scheme is selected from the set of blue light-emitting schemes, and the third green light-emitting scheme is selected from the set of green light-emitting schemes; the third blue light-emitting scheme is at least the same as the first blue light-emitting scheme and the second blue light-emitting scheme One of them is different; and/or, the third green light-emitting scheme is different from at least one of the first green light-emitting scheme and the second green light-emitting scheme; there are no two acquisitions corresponding to the same blue light-emitting scheme and the same green light-emitting scheme.

The image acquisition device for acquiring the handprint image at the third moment may be completely the same, completely different or partially the same as the image acquisition device for acquiring the handprint image at the first moment. Similarly, the image acquisition device for acquiring the handprint image at the third moment may be completely the same, completely different or partly the same as the image acquisition device for acquiring the handprint image at the second moment.

In this embodiment, a third acquisition is added. In the third acquisition, the blue light emission scheme or the green light emission scheme may be changed relative to at least one of the first two acquisitions.

According to the embodiment of the present application, the lighting scheme satisfies both the first condition and the second condition; the first condition includes: the first blue lighting scheme is the same as the second blue lighting scheme, or the first green lighting scheme is the same as the second green lighting scheme The scheme is the same; the second condition includes: the third blue light-emitting scheme is the same as the first blue light-emitting scheme, or the third blue light-emitting scheme is the same as the second blue light-emitting scheme, or the third green light-emitting scheme is the same as the first blue light-emitting scheme The green light emitting scheme is the same, or the third green light emitting scheme is the same as the second green light emitting scheme.

In the case that the first blue light-emitting scheme is the same as the second blue light-emitting scheme, the third blue light-emitting scheme is different from the first blue light-emitting scheme and the second blue light-emitting scheme, and the first green light-emitting scheme is different from the second green light-emitting scheme. The lighting schemes are different, and the third green lighting scheme is the same as the first green lighting scheme or the second green lighting scheme. Conversely, when the first green light-emitting scheme is the same as the second green light-emitting scheme, the third green light-emitting scheme is different from the first green light-emitting scheme and the second green light-emitting scheme, and the first blue light-emitting scheme is different from the second blue light-emitting scheme. The schemes are different, and the third blue light-emitting scheme is the same as the first blue light-emitting scheme or the second blue light-emitting scheme.

That is, in three acquisitions, the blue emission scheme is changed once, and the green emission scheme is changed once. In this way, it is convenient to collect handprint images under richer luminous conditions.

According to the embodiment of the present application, the number of image acquisition devices in one or more image acquisition devices is greater than 1, and all image acquisition devices are used for each acquisition.

All image acquisition devices are used to collect handprint images together during each collection, so that all collections can be completed only by sequentially emitting light according to all required lighting schemes. This collection scheme is more efficient.

According to the embodiment of the present application, the number of image acquisition devices in the one or more image acquisition devices is greater than 1, and the image acquisition devices used in at least two acquisitions among the multiple acquisitions are not exactly the same.

For example, referring to FIG. 2 , it is assumed that the image acquisition device at the center belongs to the first group of image acquisition devices, and the two image acquisition devices at the two sides belong to the second group of image acquisition devices. Images may be collected by the first group of image acquisition devices at the first moment, and images may be collected by the second group of image acquisition devices at the second moment. Although different image acquisition devices are used to collect images at different times, the handprint images collected by different image acquisition devices can still be spliced. This separate acquisition solution is convenient to avoid that the light sources corresponding to other image acquisition devices hinder the acquisition of the current image acquisition device.

According to an embodiment of the present application, one or more image acquisition devices include a first image acquisition device, a second image acquisition device and a third image acquisition device, wherein the optical axis of the first image acquisition device is aligned with the direction of the handprint acquisition area Or the planes of multiple image acquisition devices are vertical, the optical axis of the second image acquisition device forms a first preset angle with the optical axis of the first image acquisition device, the optical axis of the third image acquisition device and the first image acquisition device The optical axis forms a second preset angle; the lighting system includes one or more sets of light sources, each set of light sources includes at least one blue light source and at least one green light source, one or more sets of light sources and one or more image acquisition devices One-to-one correspondence; when the blue light source group emits light with a blue light-emitting scheme, and the green light source group emits light with a green light-emitting scheme, the images of the handprint collection area collected by one or more image acquisition devices include: at the first moment, in the blue When the color light source group emits light with the first blue light-emitting scheme, and the green light source group emits light with the first green light-emitting scheme, the image of the handprint collection area is collected by the first group of image acquisition devices; When the second blue light-emitting scheme is used to emit light, and the green light source group emits light with the second green light-emitting scheme, the image of the handprint collection area is collected by the second group of image acquisition devices, wherein the first group of image acquisition devices is the first image One of the acquisition device and the angle image acquisition device, the second group of image acquisition devices is the other in the first image acquisition device and the angle image acquisition device; the angle image acquisition device includes the second image acquisition device and the third image acquisition device , the first blue light-emitting scheme is to make only the luminous intensity of the blue light source in the blue light source group corresponding to the image acquisition device that collects at the first moment is not 0; the first green light-emitting scheme is to make only the green light source group and the second The luminous intensity of the green light source corresponding to the image acquisition device that collects at one moment is not 0; the second blue light emission scheme is to only make the luminous intensity of the blue light source corresponding to the image acquisition device that collects at the second moment in the blue light source group not equal to 0. is 0; the second green light-emitting solution is to only make the luminous intensity of the green light source in the green light source group corresponding to the image acquisition device that collects at the second moment not be 0.

The arrangements of the first image capture device, the second image capture device and the third image capture device have been described above, and will not be repeated here. In addition, the above also describes the embodiment in which the lighting system includes a light source set and the light source set corresponds to the image acquisition device one by one, so details will not be repeated here.

As mentioned above, the first image acquisition device, the second image acquisition device and the third image acquisition device can be divided into two groups, the image acquisition device located in the center is one group, and the two image acquisition devices located on both sides are another group . When any group of image acquisition devices acquires images, only the luminous intensity of the light sources in the corresponding light source set may not be 0, and the luminous intensities of the light sources in other light source sets are all 0.

Since the light source set in the center is relatively close to the light source sets on the left and right sides, it is easier to interfere with the image acquisition of the respective corresponding image acquisition devices. Emits asynchronously, that is, do not emit light at the same time. This helps to obtain a higher-precision handprint image.

In addition, the light source sets on both sides are relatively far away from each other, so the light source sets on both sides can optionally be made to emit light synchronously, which can shorten the time for handprint collection as much as possible. Of course, optionally, it is also feasible to further enable the sets of light sources located on both sides to emit light asynchronously. In this way, the interference of adjacent light sources on the image acquisition device can be avoided as far as possible.

The above scheme of making the first group of light source sets and the second group of light source sets emit light asynchronously can be applied when the light source corresponding to the first group of image acquisition devices affects the image acquisition of the second group of image acquisition devices and/or the second group of images In the case where the light source corresponding to the acquisition device affects the image acquisition of the first group of image acquisition devices.

For example and not limitation, the time difference between the first moment and the second moment may be smaller than a preset time threshold. The preset time threshold may be set to any appropriate value as required, and this application is not limited thereto. For example, the preset time threshold may be 25ms.

Considering the shaking of fingers or palms, in order to reduce the difficulty of splicing the images of different image acquisition devices together, the time interval between the two sets of light sources that emit light asynchronously should be set as short as possible. It is advisable to exceed 25ms. This asynchronous lighting method is especially suitable for packaged LED modules.

According to the embodiment of the present application, at the first moment, when the blue light source group emits light with the first blue light-emitting scheme, and the green light source group emits light with the first green light-emitting scheme, the handprint collection area is collected by the first group of image acquisition devices image, including: sending a trigger signal to the first group of image acquisition devices through the processing device at the first moment and sending a light emission control signal to the first group of light source sets corresponding to the first group of image acquisition devices; at the second moment , when the blue light source group emits light with the second blue light-emitting scheme, and the green light source group emits light with the second green light-emitting scheme, the image of the handprint collection area is collected by the second group of image acquisition devices, including: through the processing device at the second 2. Send a trigger signal to the second group of image acquisition devices at any time and send a light emission control signal to the second group of light source sets corresponding to the second group of image acquisition devices; wherein, the trigger signal is used to trigger the corresponding image acquisition device to start acquisition For images, the lighting control signal is used to control the corresponding set of light sources to start to emit light with a non-zero intensity, and the time difference between the first moment and the second moment is equal to the sum of the acquisition time of the first group of image acquisition devices and the preset delay time.

Exemplary and non-limiting, can control the map of each image acquisition device by processing device 240 Image acquisition and illumination of each light source. The processing device 240 may control the image acquisition device and the corresponding light source set to start working synchronously by simultaneously sending a trigger signal and a light emission control signal to the image acquisition device and the corresponding light source set.

In an embodiment where different groups of light source sets emit light asynchronously and different groups of image acquisition devices asynchronously capture images, the processing device 240 may first control the first group of image acquisition devices and the first group of light source sets by synchronously sending trigger signals and light emission control signals. Start to work, and then the acquisition is completed after the acquisition time of the first group of image acquisition devices, and then after a delay for a period of time (that is, the preset delay time), you can start to control the second group of images by synchronously sending trigger signals and light control signals The collection device and the second group of light sources start to work. The acquisition time of the image acquisition device is generally relatively short, such as 2ms. The delay time can be set to be small, and the sum of the delay time and the acquisition time of the first group of image acquisition devices is less than the above-mentioned preset time threshold.

According to an embodiment of the present application, the lighting system includes one or more light source sets, each light source set includes at least one blue light source and at least one green light source, and the one or more light source sets correspond to one or more image acquisition devices one by one, For any light source set in one or more light source sets, the blue light source and the green light source in the light source set emit light synchronously, and the light-emitting periods of the blue light source and the green light source in the light source set are within the effective period of the corresponding image acquisition device. within the exposure period.

The luminous period of the light source refers to the period when the luminous intensity of the light source is not zero. The exposure mode of any image acquisition device in one or more image acquisition devices may adopt global exposure or line exposure. In the case that the current image acquisition device adopts global exposure, the exposure period of all lines in each frame of image captured by the current image acquisition device is uniform, and the uniform exposure period may be called an effective exposure period. The light-emitting period of each light source in the light source set corresponding to the current image acquisition device only needs to be within the effective exposure period. The light-emitting period of the light sources in the light source set corresponding to the current image capture device may be shorter than or equal to the effective exposure period during global exposure.

In the case that the current image acquisition device adopts row exposure, the effective exposure period may be an exposure period shared by all the rows in each frame of image. Line exposure can be further divided into at least two exposure modes, one is line exposure with initial time synchronization, and the other is line exposure without initial time synchronization.

Fig. 17 shows a schematic diagram of image acquisition and light emitting sequence according to an embodiment of the present application. The row exposure mode shown in FIG. 17 is synchronized with the initial time, that is, the initial time of the exposure of each row is the same, but the end time is different. For example, the total exposure time is 15 seconds, the exposure time of the first line is 1-10 seconds, the exposure time of the second line is 1-11 seconds, the exposure time of the third line is 1-12 seconds... Exposure times for the six rows are 1-15 seconds. The effective exposure time at this time is 1-10 seconds, so the lighting period of the light source can be set within 1-10 seconds. The total duration during which the luminous intensity of the light source is not 0 may be equal to or shorter than 10 seconds.

For row exposure without initial time synchronization, only the initial time is different from the example shown in FIG. 17 , and other times, such as the end time of each row exposure and data output time, are similar. For example, if the initial time synchronization is not performed, and the total exposure time is 15 seconds, the exposure time of the first row The exposure time of the second row is 2-11 seconds, the exposure time of the third row is 3-12 seconds... the exposure time of the sixth row is 6-15 seconds. The effective exposure time at this time is 6-10 seconds, so the lighting period of the light source can be set within 6-10 seconds. The total duration during which the luminous intensity of the light source is not 0 may be equal to or shorter than 5 seconds.

The light-emitting period of the blue light source and the green light source in the light source set is within the effective exposure period of the corresponding image acquisition device, so that different positions (for example, different rows) on the image collected by the image acquisition device are under the same lighting conditions Acquisition is obtained, so as to ensure the consistency of the imaging conditions of the acquired images, which helps to improve the effect of subsequent fusion stitching or splicing fusion.

According to an embodiment of the present application, the lighting system includes one or more light source sets, each light source set includes at least one blue light source and at least one green light source, and the one or more light source sets correspond to one or more image acquisition devices one by one, Methods also include:

For each of the one or more image capture devices,

Extracting the blue channel and the green channel from the same handprint image collected by the image acquisition device to obtain the current blue channel image and the current green channel image;

Calculating the respective contrast and/or brightness of the current blue channel image and the current green channel image;

Based on the contrast and/or brightness of the current blue channel image and the contrast and/or brightness of the current green channel image, determine the adjusted blue luminous intensity and the adjusted green luminous intensity of the light source set corresponding to the image acquisition device, and adjust the blue luminous intensity. and adjusting the luminous intensity of the green light so that the contrast and/or brightness of the subsequent blue channel image collected by the image acquisition device under the adjusted luminous intensity of the blue light and the subsequent green channel image collected under the adjusted luminous intensity of the green light are consistent;

Controlling the blue light source in the light source set corresponding to the image acquisition device to emit blue light to the handprint collection area according to the adjusted blue light luminous intensity;

The green light source in the light source set corresponding to the image acquisition device is controlled to emit green light to the handprint collection area according to the adjusted green light emission intensity.

The present embodiment will be described below by taking three image acquisition devices and three light source sets as examples. For example, the images of the blue channel and the green channel in the images currently captured by the three image acquisition devices can be extracted in real time, and the contrast and/or the corresponding blue channel images and green channel images of the three image acquisition devices can be calculated. or brightness. For example, the respective brightness values can be obtained by calculating the histogram of the blue channel image and the histogram of the green channel image. Subsequently, the adjusted blue light luminous intensity corresponding to each of the three light source sets corresponding to the three image acquisition devices may be calculated according to the contrast and/or brightness of the three blue channel images corresponding to the three image acquisition devices. In addition, the adjusted green luminous intensity corresponding to each of the three light source sets corresponding to the three image acquisition devices can be calculated according to the contrast and/or brightness of the three green channel images corresponding to the three image acquisition devices. Then, when emitting light next time, control each of the three light source sets to adjust the blue light luminous intensity and adjust the green light according to the calculation above. Luminous intensity glows.

The above embodiments can independently adjust the light source intensity of a single channel. By adjusting the blue and green light sources separately, the light sources of the same color at different collection positions can have relatively consistent contrast and/or brightness, which facilitates the subsequent stitching of images at different collection positions to obtain a better stitching effect. In addition, by independently adjusting the light source intensity of a single channel, the luminous intensity of the current blue-green light source can be adjusted based on the skin's response to light to better adapt to different skin conditions, thereby further adapting to more people.

According to the embodiment of the present application, when the blue light source group emits light in a blue light-emitting scheme, and the green light source group emits light in a green light-emitting scheme, collecting images of the handprint collection area through one or more image acquisition devices includes: at any target light-emitting moment , when the blue light source group emits light with a selected blue light-emitting scheme, and the green light source group emits light with a selected green light-emitting scheme, at least part of the image capture devices in one or more image capture devices are used to collect images of the handprint collection area; Among them, the selected blue light-emitting scheme is selected from the set of blue light-emitting schemes, and the selected green light-emitting scheme is selected from the set of green light-emitting schemes; the set of blue light-emitting schemes includes w kinds of blue light-emitting schemes, where w∈{1,2 ,3,...,N _t1 -1,N _t1 }, N _t1 is to select a gear from the N _i gears of the luminous intensity corresponding to the i-th blue light source in the blue light source group. The total number of combinations obtained by combining the gears of the luminous intensity of the blue light sources in the color light source group, N _i is the total number of gears of the luminous intensity of the i-th blue light source; i=1,2,3..., k1-1, k1; k1 is the total number of blue light sources in the lighting system; the set of green lighting schemes includes p green lighting schemes, where p∈{1,2,3,…,N _t2 -1,N _t2 }, N _t2 is to combine the gears of the luminous intensity of the green light sources in the green light source group by selecting a gear from the N _j gears of the luminous intensity corresponding to the jth green light source in the green light source group The total number of combinations obtained, N _j is the total number of stalls of the luminous intensity of the jth green light source; j=1,2,3...,k2-1,k2; k2 is the total number of green light sources in the lighting system .

The above describes an embodiment where the luminous intensities of the blue light source and the green light source are respectively divided into two levels, but this is only an example. The total number of gears of the luminous intensity of the i-th blue light source can be expressed as N _i , where the corresponding N _i of any two different blue light sources can be equal or unequal, and any blue light source The corresponding N _i may be greater than or equal to 2. For example, the luminous intensity of a certain blue light source can be divided into 3 levels, and the luminous intensity of another blue light source can be divided into 4 levels. The gears of the luminous intensity of all blue light sources can be combined. For example, assuming that the lighting system includes three blue light sources, the luminous intensity of the first blue light source is divided into 2 levels, the luminous intensity of the second blue light source is divided into 4 levels, and the luminous intensity of the third blue light source is divided into 4 levels. Divided into three levels, the total number of combinations N _t1 can be equal to 2×4×3, that is, equal to 24. Similarly, the total number of gears of the luminous intensity of the jth green light source can be expressed as N _j , where N _j corresponding to any two different green light sources can be equal or unequal, and any green light source The corresponding N _j may be greater than or equal to 2. The combination of the luminous intensity of the green light source is similar to the combination of the blue light source, and will not be repeated here.

It can be understood that, when the target light-emitting time is the above-mentioned first time, the blue light-emitting scheme is selected as the first blue light-emitting scheme, and the green light-emitting scheme is selected as the first green light-emitting scheme. In the eyes When the marked lighting time is the above-mentioned second time, the blue lighting scheme is selected as the second blue lighting scheme, and the green lighting scheme is selected as the second green lighting scheme. When the target lighting time is the third time, the blue lighting scheme is selected as the third blue lighting scheme, and the green lighting scheme is selected as the third green lighting scheme.

According to an embodiment of the present application, when there is a blue light source whose luminous intensity is not 0 in the blue light source group, the handprint images collected include at least one first handprint image, and any two of the at least one first handprint image The blue light-emitting schemes adopted for the collection of the first handprint images are different from each other, and the handprint collection method further includes: selecting a first handprint image whose image quality meets the first target requirement from at least one first handprint image for use in Perform blue-green fusion operation. When there is a green light source whose luminous intensity is not 0 in the green light source group, the handprint images collected include at least one second handprint image, and any two second handprint images in the at least one second handprint image are collected The adopted green lighting schemes are different from each other, and the handprint collection method further includes: selecting a second handprint image whose image quality meets the second target requirement from at least one second handprint image for performing a blue-green fusion operation.

The first target requirement and the second target requirement may be the same or different. In one or some embodiments, image quality evaluation may be performed on each first handprint image to obtain a quality score of each first handprint image. The first target requirement may include that the quality score of the first handprint image is greater than a first quality score threshold. The first quality scoring threshold can be set to any size as required. The image quality evaluation may include evaluating one or more of the clarity of the first handprint image, the degree of occlusion of the three-dimensional object (part or all of the user's hand), the size of the three-dimensional object, and the like. In one or some embodiments, image quality evaluation may be performed on each second handprint image to obtain a quality score of each second handprint image. The second target requirement may include that the quality score of the second handprint image is greater than a second quality score threshold. The second quality scoring threshold can be set to any size as required. The image quality evaluation may include evaluating one or more of the clarity of the second handprint image, the degree of occlusion of the three-dimensional object (part or all of the user's hand), the size of the three-dimensional object, and the like.

For example, assuming that the blue light source group includes 3 blue light sources in total, and the luminous intensity of each blue light source is divided into 3 levels, there are a total of 27 blue luminous schemes, except that the luminous intensity of all blue light sources is 0 In this case, there are 26 remaining blue light-emitting schemes. In the case of using these 26 kinds of blue light-emitting schemes to emit light, 26 first handprint images can be collected respectively, and the first handprint image whose image quality meets the first target requirement can be selected from the 26 first handprint images. Used to perform blue-green fusion operations. The selection method for the second handprint image is similar and will not be repeated here.

According to the embodiment of the present application, the handprint collection method may further include: performing a blue-green fusion operation. The blue-green fusion operation may include: extracting the blue channel and the green channel from the same handprint image collected by the same image acquisition device to obtain the blue channel image and the green channel image; fusing the blue channel image and the green channel image to obtain the fused handprint image corresponding to the image acquisition device. In this case, the first handprint image and the second handprint image can be the same handprint picture.

There are three basic channels of the original color image collected by the image acquisition device: red, green, and blue (RGB). By processing the image into a matrix, the value of the corresponding channel can be obtained.

After extracting the grayscale images of the green channel and the blue channel respectively, the grayscale images of these two channels can be fused together. The fusion operation may be implemented in any suitable fusion manner, and this paper will describe two exemplary fusion manners below.

According to the embodiment of the present invention, the blue channel image, the green channel image and the fused handprint image are each divided into m units, where m is an integer greater than 1,

The fusion of the blue channel image and the green channel image includes:

From the ith unit of the blue channel image or the further processed blue channel image and the ith unit of the green channel image or the further processed green channel image, selecting the unit whose pixel value fluctuates the most;

Determine the pixel value of the i-th unit of the fusion handprint image based on the pixel value of the selected unit;

Among them, i=1,2,3...m, the further processed blue channel image is the image obtained by further processing the blue channel image, and the further processed green channel image is obtained by further processing the green channel image Image.

m can be set to any suitable value as required, and the present invention is not limited thereto. The m units can be divided in any suitable manner. In one example, the m units can be divided into pixels, that is, each pixel is divided into a unit, and the image can be divided into as many units as there are pixels. In another example, every plurality of pixels in the image is divided into a unit.

The degree of pixel value fluctuation can be expressed in many ways, which mainly reflects how severe the pixel change at the current unit in the image is. For any unit, on the fused handprint image, always try to keep the pixel information from the unit whose pixel value fluctuates more violently in the blue channel image and the green channel image. The pixel contrast on the fused handprint image obtained in this way is strong, which is convenient for subsequent comparison between handprints when the handprint image is used for handprint recognition.

According to the embodiment of the present invention, each unit in the m units only includes a single pixel, and the fluctuation degree of the pixel value of any unit is represented by the absolute value of the difference between the pixel value of the unit and the average pixel value of surrounding pixels, further Processing includes normalization,

The fusion of the blue channel image and the green channel image includes:

Normalize the pixel values of the blue channel image and the green channel image;

For the pixel a ₁ contained in the i-th unit in the first channel image, calculate the average pixel value E ₁ of M pixels around the pixel, where M is an integer greater than 1;

For the pixel a ₂ contained in the i-th unit in the second channel image, calculate the average pixel value E ₂ of the M pixels around the pixel;

If |F ₁ -E ₁ |>|F ₂ -E ₂ |, then determine F ₃ =F ₁ -E ₁ +(E ₁ +E ₂ )/2;

Wherein, the first channel image and the second channel image are respectively one of the normalized blue channel image and the normalized green channel image, and the first channel image and the second channel image are different, F ₁ is the pixel a ₁ pixel value, F ₂ is the pixel value of pixel a ₂ , F ₃ is the pixel value of the pixel contained in the i-th unit in the fused handprint image.

This embodiment belongs to a pixel-by-pixel fusion method. Since the pixel values of the grayscale images of different channels need to be summed, the grayscale images of the two channels can be normalized first. Normalization can be achieved using any suitable normalization method.

M can be set to any suitable value as required, and the present invention is not limited thereto. The following describes this embodiment by taking M=8 as an example. For example, the pixel value _F1 of pixel _a1 can be obtained from the normalized blue channel image, and the pixel value F of pixel _a2 at the position corresponding to pixel _a1 can be obtained from the normalized green channel image ₂ . Subsequently, the average pixel value E ₁ of the 8 pixels around a ₁ can be calculated, and the average pixel value E ₂ of the 8 pixels around a ₂ can be calculated. If |F ₁ -E ₁ |>|F ₂ -E ₂ |, use F ₁ -E ₁ +(E ₁ +E ₂ )/2 as the pixel corresponding to a ₁ or a ₂ in the fused image The final pixel value of . Those skilled in the art can understand that for a pixel with less than 8 surrounding pixels (for example, the first pixel in the upper left corner of the image), the average pixel value can be calculated by taking the pixel values of all the surrounding pixels of the pixel.

According to an embodiment of the present invention, normalizing the pixel values of the blue channel image and the green channel image includes: obtaining the first pixel value at a specific percentage of all pixel values of the blue channel image and all pixel values of the green channel image the second pixel value at a specific percentage in ; unify the first and second pixel values to the same specific pixel value; unify all pixels in the blue channel image based on the ratio between the first pixel value and the specific pixel value value to obtain a normalized blue channel image; all pixel values in the green channel image are scaled based on the ratio between the second pixel value and the specified pixel value to obtain a normalized Optimized green channel image.

The particular percentage can be any suitable percentage value, without limitation herein. In one example, the specific percentage is 20%. The following takes this as an example to describe the way of normalization. For example, the first pixel value can be obtained by taking the pixel value at the top 20% quantile from the blue channel image, and the second pixel value can be obtained by taking the pixel value at the top 20% quantile from the green channel image. It can be understood that the above-mentioned top 20% quantile refers to the pixel value whose pixel value is at the 20% position among all the pixel values of the entire image.

Subsequently, the first pixel value and the second pixel value may be unified to an equal specific pixel value. For example, assuming that the first pixel value is 200 and the second pixel value is 100, the first pixel value can be reduced to half of its original value to 100, so that the first pixel value and the second pixel value are uniformly 100. Then, all the pixel values in the blue channel image are reduced to half of the original value, and the pixels in the green channel image are unchanged, so that the normalization of the two channels can be completed.

Of course, the above scales down the pixel values of the blue channel image while the pixel values of the green channel image do not The variant embodiments are only examples and do not limit the present invention. The present invention can also adopt other suitable normalization methods. For example, it is possible that the pixel values of the blue channel image are unchanged and the pixel values of the green channel image are enlarged, or that the pixel values of both the blue channel image and the green channel image are changed.

According to an embodiment of the present invention, each of the m units contains a plurality of pixels, and the fluctuation degree of the pixel value of any unit is represented by the mean square error or gradient of the unit, and the fusion of the blue channel image and the green channel image includes: Calculate the mean square error or gradient of each of the m units in the blue channel image; calculate the mean square error or gradient of each of the m units in the green channel image; from the i-th unit in the blue channel image In the ith unit in the unit and the green channel image, select the unit with a larger mean square error or gradient; determine the pixel value of the pixel contained in the selected unit as contained in the ith unit in the fusion handprint image The pixel value of the pixel.

In addition to pixel-by-pixel fusion, block fusion is also possible. For example, each cell divided for an image may contain a plurality of pixels. For the corresponding blocks in the blue channel image and the green channel image, the mean square error or gradient of the respective blocks can be calculated, and the block with the largest mean square error or gradient in the block is selected from the corresponding blocks of the blue channel image and the green channel image as The final blocks in the handprint image are fused.

According to an embodiment of the present invention, the method further includes: smoothing the pixel values at the junction of adjacent units in the fused handprint image to obtain an updated fused handprint image.

The junction between blocks can be smoothed, otherwise the border will be more abrupt. The smoothing process may be any suitable smoothing process, which is not limited herein. For example, an average operation may be performed on pixel values of a preset number of pixels at the junction of two or more blocks, and the average value may be used as a new pixel value of the preset number of pixels.

Exemplarily, the handprint collection method may further include: for each of the one or more image collection devices, performing the above-mentioned blue-green fusion operation; pattern image.

The corresponding fused handprint image can be obtained by fusing the blue channel image and the green channel image of any image acquisition device. Exemplarily, when the number of the image acquisition device is one, the fused handprint image corresponding to the image acquisition device may be used as the overall handprint image. Exemplarily, when there are multiple image acquisition devices, the fused handprint images corresponding to the multiple image acquisition devices may be spliced together to obtain an overall handprint image. The overall handprint image can be used for subsequent purposes such as handprint recognition.

Most of the existing non-contact collection devices are designed to collect single-finger fingerprints, and there are no non-contact collection devices that require large collection areas such as four-finger fingerprints, double-thumb fingerprints, flat palm palmprints, and side palm palmprints. . If the existing non-contact collection device for collecting single-fingerprints is directly used to collect one or more of the aforementioned large-area handprints, the desired collection of handprints will be incomplete, or the imaging quality of the peripheral field of view will be lower than The central field of view, or the acquisition effect is not ideal due to different acquisition accuracy requirements.

The embodiment of the present application provides a handprint collection system. Referring to FIG. 18 , the handprint collection system includes a structured light projector 100 , a plurality of fourth image collection devices 1820 and a processing device (not shown). The handprint collection system described in this article can collect handprints including but not limited to single-fingerprints, double-thumbprints, four-fingerprints, flat-palm palmprints and side-palm palmprints, and is especially suitable for collecting large-area palmprints. Referring to FIG. 19 , the structured light projector 100 can be aimed at the handprint collection area, and is used for emitting structured light to the handprint collection area. The dotted rectangular box A in FIG. 19 exemplarily marks the handprint collection area. The handprints expected to be collected may face the structured light projector 100 and the plurality of fourth image collection devices 1820 . Although the handprint collection area shown in the figure is located above the structured light projector 100 and a plurality of fourth image acquisition devices 1820, in other embodiments not shown, the handprint collection area may also be located on the structured light projector 100 and the front side of a plurality of fourth image capture devices 1820 .

Exemplarily, the structured light projected by the structured light projector 100 may be film projection structured light. The structured light projector 100 may be located on the side closer to the base of the palm. In this way, the structured light projector 100 can be prevented from occupying the central area of the entire acquisition device, and image acquisition devices can be arranged in the central area, such as a plurality of fourth image acquisition devices 1820, a plurality of fifth image acquisition devices 1830 and /or the sixth image acquisition device 1840 to be mentioned below. The arrangement of the above-mentioned structured light projector 100 is only an example, and it can also be arranged at any position in the peripheral area other than the central area. If allowed, the structured light projector 100 can also be arranged in the central area. When the structured light projector 100 is arranged in the peripheral area, it is required that the projection direction of the structured light and the collection surface (handprint surface) form a certain angle. Only in this way can the projection range of the structured light cover the entire handprint collection area.

Since the projection direction of the structured light is at a certain angle to the collection surface, in order to avoid the problem that the farthest end and the nearest end of the collection surface are on both sides of the projected image plane, and the structured light is out of focus, the pattern generation unit can be combined with the converging The optical axis of the lens is arranged at a certain angle, rather than being arranged vertically, so that the point on the pattern generating unit closer to the converging lens forms a projected point on the collecting surface that is farther away from the converging lens. In this way, the image plane formed by the pattern on the pattern generating unit refracted by the converging lens can overlap with the collection surface as much as possible, so that the projected pattern can be as clear as possible. In this way, the out-of-focus of structured light can be avoided, and high-precision three-dimensional reconstruction of the scene can be performed.

Since the structured light projector 100 is placed obliquely towards the center of the handprint collection area, the structured light projector 100 can be placed on the side close to the user, such as the side close to the root of the palm, to avoid the light emitted by the light source from irradiating the user.

The field of view of the plurality of fourth image capture devices 1820 can cover the entire handprint collection area, so as to facilitate the collection of images of the entire handprint collection area. After the user places a finger or/or palm on the palmprint collection area, the multiple fourth image capture devices 1820 can at least clearly capture the user's fingerprint and/or palmprint images facing the multiple fourth image capture devices 1820 .

Exemplarily, the lenses of the multiple fourth image capture devices 1820 may be located in a predetermined plane, and the multiple fourth image capture devices 1820 may be respectively aimed at multiple first sub-regions in the handprint capture area. The plurality of fourth image capture devices 1820 may be spaced apart from each other by a certain distance. However, much Any two adjacent ones of the first sub-areas overlap or adjoin each other, so that multiple fourth image capture devices 1820 cooperate with each other to capture images in the entire handprint capture area. The optical axes of the multiple fourth image capture devices 1820 may be arranged parallel to each other. That is, the plurality of fourth image acquisition devices 1820 respectively acquire images in the plurality of first sub-regions at the same angle. Referring to FIG. 18 , there are three fourth image acquisition devices 1820 in the embodiment of the present application, and they are arranged in a predetermined plane at the same distance from each other. It can be understood that, in an unshown embodiment, the number of the first image acquisition device can be arbitrary, and can be specifically set according to different usage requirements of the handprint acquisition system and different parameters of the fourth image acquisition device . The plurality of fourth image acquisition devices 1820 may be respectively used to acquire images of the plurality of first sub-regions to obtain the plurality of first target images. Specifically, the obtained multiple first target images may respectively correspond to images of different parts of the collected user's fingerprint and/or palmprint, that is to say, two adjacent first sub-regions are adjacent, rather than overlapping. Alternatively, there may be partial overlap between the multiple first target images. The processing device may process the plurality of first target images to obtain images of the first type of handprint. The first type of palmprints may include at least one of four-fingerprints, double-thumbprints, flat-palm palmprints, and side-palm palmprints. Specifically, the plurality of first target images may be a subset of handprint images. The present application is provided with a plurality of fourth image acquisition devices 1820 to respectively acquire images of a plurality of first sub-regions, so as to obtain a plurality of first target images in a more targeted manner and ensure the clarity of the first target images. The processing device may perform processing such as expansion and splicing on the multiple first target images, and the obtained handprint image after processing is a combination of the multiple first target images. Specifically, in the present application, the processing device may process the obtained multiple first target images, and during processing, may splice better imaged parts of the multiple first target images to obtain a complete handprint image. In this way, each part of the handprint image obtained after processing has better definition and higher imaging quality. In addition, with the cooperation of structured light, multiple fourth image acquisition devices 1820 can acquire the collected three-dimensional shape data of single finger, double thumb, four fingers, flat palm or side palm, and perform unfolding processing to eliminate the blurring of hand lines. Obtain high-quality planar texture images due to surface undulations.

The handprint collection system of the embodiment of the present application may be non-contact, that is, the user can collect part or all of the hand over the handprint collection area without touching the collection device. In this way, the problems of high hygiene risk, low collection quality, small collection area, sensitivity to dryness and humidity of the skin, and low collection consistency caused by contact collection can be avoided. Or, optionally, the handprint collection system can also, for example, run on a smart device and realize handprint collection by touching the screen with the hand. In this case, the handprint collection system is not in contact with the hand, but the screen. Of course, the handprint collection system can also be used to collect handprint images without touching the screen.

The handprint collection system provided in the embodiment of the present application utilizes multiple fourth image collection devices whose lenses are located in the same plane to respectively collect images of different multiple first sub-regions in the handprint collection area, and process these images through the processing device Image processing can obtain a complete handprint image, so the collection of large-area handprints is especially suitable for collecting images of four-fingerprints, double-thumbprints, flat palms and side palms. Certainly, also can utilize this handprint collecting system to gather the image of single-finger fingerprint, in this case, finger can be placed at any position in the handprint collecting area, with better customer experience. Regardless of whether it is for large-area handprint collection or for handprint collection with a single finger placed more casually, a complete and high-quality handprint image can be obtained.

Exemplarily, the distance from each of the plurality of fourth image capture devices 1820 to the handprint capture area is between 72mm-82mm. In this way, it can be ensured that the handprint collection area is within the range of better imaging effect of the fourth image collection device 1820, the definition of multiple first target images can be ensured, and the imaging quality of the handprint images can be ensured. Further, the distance between each of the plurality of fourth image capture devices 1820 and the handprint capture area may be 76mm, so as to achieve better imaging effect.

Exemplarily, the focal length of each of the plurality of fourth image capture devices 1820 may be between 4-4.2mm. The focal length can be understood as the distance between the focal point of the lens in the lens of the fourth image acquisition device 1820 and the rear main plane. The size of the focal length can affect the proportion of the captured target in the captured picture. Setting the focal length in this way can further ensure that the handprint collection area is within the imaging range of the fourth image collection device 1820, and make the proportion of the handprint image in the first target image more moderate.

Exemplarily, the f-number of each of the plurality of fourth image capture devices 1820 is not less than 4. The aperture number, also known as the F-number, is the ratio of the focal length of the camera lens of each camera to the diameter of the entrance pupil of the camera lens. Larger f-numbers correspond to smaller apertures, increasing depth of field and therefore room for finger movement, while still maintaining sharpness in imaging. In the present application, the aperture number of each of the multiple fourth image acquisition devices 1820 is not less than 4, so as to ensure the size of the space for the movement of the collected fingers or palms, and within this range, the handprint collection system can collect clear hand prints. Texture images to improve user experience. Further, the f-number of each of the plurality of fourth image capture devices 1820 may not be less than 5.6.

Exemplarily, the depth of field of each of the plurality of fourth image capture devices 1820 is not less than 20mm. In this way, the thickness of the finger or palm and the height difference of surface fluctuations can be better adapted to ensure the imaging quality.

Exemplarily, the optical axes of the plurality of fourth image capture devices 1820 may be located in a first plane perpendicular to the predetermined plane. That is, the lenses of the plurality of fourth image capture devices 1820 are arranged in parallel with each other, and are sequentially arranged on the same straight line. The structure setting is more compact and easy to install. Moreover, it is possible to effectively prevent the view field blocking among the multiple fourth image capture devices 1820 .

Exemplarily, the field of view of each of the plurality of fourth image capture devices 1820 may be greater than or equal to 125×70 mm. Wherein, a plurality of fourth image capture devices 1820 may be adjacent to or overlapped on a long side corresponding to 125 mm. Specifically, taking the three fourth image acquisition devices 1820 shown in the figure as an example, the long sides of the field of view ranges of the three fourth image acquisition devices 1820 can be adjacent to each other, thus forming a maximum 125×210 mm Field of view to suit most application scenarios. Such a field of view can ensure that the combined viewing angles of multiple fourth image acquisition devices 1820 can cover a sufficiently large handprint collection area and ensure the integrity of the handprint image.

Exemplarily, the diagonal field of view of each of the plurality of fourth image capture devices 1820 may be between 80°-82°. In this way, after a plurality of fourth image acquisition devices 1820 are arranged reasonably After that, the first sub-region can be covered more comprehensively. Prevent the occurrence of dead angle areas that cannot be collected, and further ensure the integrity of the collected handprint images.

Exemplarily, the working wavelength of each of the plurality of fourth image capture devices 1820 may correspond to the wavelength of visible light. In this way, the fourth image collection device 1820 can directly collect the visible light information of the handprint collection area, and the application range of the handprint collection system is wider. Exemplarily, each of the plurality of fourth image capture devices 1820 may not include an infrared filter. Clearer handprint images can be obtained without setting infrared filters, and the structure setting is simpler and more reasonable.

Exemplarily, the resolution of each of the plurality of fourth image capture devices 1820 may be no less than 5 million pixels. Such setting of resolution can ensure the sharpness of each of the plurality of first target images, thereby ensuring the overall sharpness of the handprint image.

Exemplarily, the optical distortion of each of the plurality of fourth image capture devices 1820 may be less than 5%. Optical distortion is the degree of distortion of the image called by the optical system to the object relative to the object itself. In this application, the optical distortion of each of the plurality of fourth image acquisition devices 1820 is less than 5%, which can better ensure the authenticity of each first target image, and further ensure the authenticity of the handprint image.

Exemplarily, the interface of the lens of the fourth image acquisition device 1820 may be an M12 interface. In this way, the lens size is smaller and the structure is more compact.

Exemplarily, referring to FIG. 18 , the handprint collection system may further include a plurality of fifth image collection devices 1830 . The lenses of at least a part of the multiple fifth image capture devices 1830 are located in different planes and aligned with multiple second sub-regions in the handprint capture area. In this way, these fifth image acquisition devices 1830 can better acquire handprint information on different planes, so as to obtain a full single-fingerprint image. Full single-finger fingerprints include fingerprints on the front and two sides of the pad of a single finger. The plurality of fifth image acquisition devices 1830 are respectively used to acquire images of the plurality of second sub-regions to obtain a plurality of second target images, and the processing device is also used to process the plurality of second target images to obtain a second type of target image. patterned image. The second type of fingerprints may include full single-finger prints. Wherein, the plurality of second sub-regions may be located in different planes from each other. Further, the plurality of second sub-regions may be spatially separated from the plurality of first sub-regions. Alternatively, the plurality of second regions may have partial overlap with the plurality of first subregions. Multiple fifth image capture devices 1830 may be used to capture 3D images of fingerprints, and multiple second sub-regions may correspond to the front and side regions of the finger, respectively. The processing device may stitch the fingerprint images of the plurality of second sub-regions onto the 3D fingerprint surface of the 3D fingerprint model to obtain a 3D fingerprint image. With the cooperation of the fifth image acquisition device 1830 and the structured light projector, the area of the fingerprint obtained is much larger than that obtained by traditional contact, and the acquisition quality is higher. When a high-quality, full single-fingerprint is required, you only need to place the finger in the image acquisition area and hover for a moment to obtain a high-definition and complete single-fingerprint image. There is no need to roll a finger from side to side on the capture device to increase the total contact area. This solution is especially suitable for individuals who are not compliant, and there will be no blurring of fingerprint images.

The plurality of fifth image capture devices 1830 may be or include the above-mentioned first image capture device, second image capture device and third image capture device.

The handprint collection system includes a plurality of fourth image collection devices 1820 and a plurality of fifth image collection devices 1830. If it is necessary to collect high-quality single-fingerprints, multiple fifth image collection devices 1830 can be used for collection. In the case of a large-area fingerprint or a single-finger side fingerprint, a plurality of fourth image capture devices 1820 may be used for capture. A plurality of fourth image acquisition devices 1820 and a plurality of fifth image acquisition devices 1830 may not work at the same time, and the structured light projector 100 may operate when a plurality of fourth image acquisition devices 1820 and a plurality of fifth image acquisition devices 1830 are in operation Both cast structured light. Therefore, it can meet various collection requirements such as single finger, four fingers, double thumbs, flat palm and side palm, which makes the application range of the handprint collection system wider.

Exemplarily, referring to FIG. 20 , the plurality of fifth image capture devices 1830 may include a middle fifth image capture device 1833 , a left fifth image capture device 1831 and a right fifth image capture device 1832 . Wherein, the fifth image capture device 1831 on the left is located on the left side of the fifth image capture device 1833 in the middle. The fifth image capture device 1832 on the right is located on the right side of the fifth image capture device 1833 in the middle. The optical axes of the lenses of the left fifth image capture device 1831 and the right fifth image capture device 1832 are tilted at a set angle, respectively facing the corresponding second sub-regions. Exemplarily, the left fifth image capture device 1831 and the right fifth image capture device 1832 may be symmetrically arranged on both sides of the middle second capture device 1833 . When the fifth image capture device 1830 is working, it can capture images of the middle, left and right regions of the target respectively. This setting is especially suitable for single-finger collection. The second sub-region can include the fingerprints on the front, left and right sides of the pad of the finger, so that a clearer and more accurate 3D fingerprint image of a single finger can be obtained.

Exemplarily, the lens of the middle fifth image capture device 1833 may be located in a predetermined plane. In this way, the fifth image capture device 1833 in the middle and the multiple fourth image capture devices 1820 are arranged on the same plane, the structure is more compact, and the field of view between the fifth image capture device 1833 and the multiple fourth image capture devices 1820 is prevented from being blocked .

Of course, in other embodiments not shown, two or more fifth image acquisition devices 1830 may also be provided. When only two fifth image capture devices 1830 are provided, the two fifth image capture devices 1830 may be arranged at the lower left and lower right of the single finger. When more fifth image capture devices 1830 are provided, the arrangement is more flexible.

Exemplarily, the distance from each of the plurality of fifth image capture devices 1830 to the handprint capture area is between 72mm-82mm. Such setting can ensure that the handprint collection area is within the better imaging range of the fifth image collection device 1830 and ensure the clarity of multiple second target images. Further, the distance between each of the plurality of fifth image capture devices 1830 and the handprint capture area may be 76 mm to achieve better imaging effect.

Exemplarily, the focal length of each of the plurality of fifth image capture devices 1830 may be between 5-7mm. It is further ensured that the handprint collection area is within the better imaging range of the fifth image collection device 1830 to ensure the imaging effect of the handprint collection system. Moreover, the proportion of the image of the handprint in the second target image is more moderate. Further, the focal length of each of the fifth image capture devices 1830 may be 6mm.

Exemplarily, the field of view of each of the plurality of fifth image capture devices 1830 is greater than or equal to 40×40 mm. In this way, it can be ensured that the field of view ranges of the plurality of fifth image capture devices 1830 can cover a sufficiently large handprint capture area.

Exemplarily, the diagonal field of view of each of the plurality of fifth image capture devices 1830 is between 61°-63°. In this way, the multiple fifth image capture devices 1830 can cover multiple second sub-areas more comprehensively after being arranged, preventing the occurrence of dead angle areas that cannot be captured. Preferably, the diagonal field of view of each of the fifth image capture devices 1830 may be 62°.

Exemplarily, the depth of field of the plurality of fifth image capture devices 1830 is not less than 20mm. In this way, it can better adapt to the thickness of fingers or palms, as well as the height difference of surface undulations, so as to ensure the imaging quality.

Exemplarily, the range of field of view of each of the plurality of fifth image capture devices 1830 may be smaller than the range of field of view of each of the fourth image capture devices 1820 . In this way, the fifth image acquisition device 1830 can be used to capture a smaller second sub-region, and the fourth image acquisition device 1820 can be used to capture a relatively larger first sub-region. More detailed and more targeted image collection can be carried out in the handprint collection area, and the collection quality is higher.

Exemplarily, the focal length of each of the plurality of fifth image capture devices 1830 may be greater than the focal length of each of the plurality of fourth image capture devices 1820 . In this way, the second target image captured by the fifth image capture device 1830 has a larger proportion of the subject (ie, handprint) than the first target image captured by the fourth image capture device 1830 . The fifth image capture device 1830 can better capture fingerprints of smaller subjects such as fingers. The fourth image capture device 1820 can capture a relatively large capture subject such as palm prints of a palm.

Exemplarily, the diagonal field of view of each of the plurality of fifth image capture devices 1830 may be smaller than the diagonal field of view of each of the fourth image capture devices 1820 . That is, the fifth image capture device 1830 has a smaller field of view than the fourth image capture device 1820 . In this way, the fifth image collection device 1830 collects smaller collection targets, which can reduce interference and ensure the collection quality of all single-fingerprint images. The fourth image capture device 1820 can be used to capture larger capture objects. The larger field of view can ensure the integrity of the handprint image.

Exemplarily, the optical axes of the plurality of fourth image acquisition devices 1820 are located in a first plane perpendicular to the predetermined plane, the optical axes of the plurality of fifth image acquisition devices 1830 are located in a second plane perpendicular to the predetermined plane, and the first The plane is parallel to the second plane. In this way, the center of the first sub-region captured by the fourth image capture device is spaced a certain distance from the center of the second sub-region captured by the fifth image capture device. That is, the handprint collection area can be divided into a first sub-area and a second sub-area, and the two can be set independently of each other, and can also have partial overlapping areas. In the corresponding area, more targeted handprint collection can be performed to further ensure the quality of image collection within the range of each sub-area.

Exemplarily, referring to FIGS. 18-20 , the handprint collection system may further include a sixth image collection device 1840 . The field of view of the sixth image capture device 1840 covers the entire handprint capture area. In this way, through The image of the handprint collection area can be obtained in real time through the sixth image collection device 1840 . The handprint collection system may be connected with a display device or the handprint collection system itself includes a display device. The images in the handprint collection area collected by the sixth image collection device 1840 can be displayed in real time by the display device to serve as a preview. It can guide users to place one finger, four fingers, two thumbs, palm or side palm in the appropriate area to improve user experience.

Exemplarily, the lens of the sixth image capture device 1840 may be located in a predetermined plane. In this way, the sixth image capture device 1840 can be located in the same plane as the plurality of fourth image capture devices 1820 , the structure is compact, and the field of view between the sixth image capture device 1840 and the multiple fourth image capture devices 1820 is prevented from being blocked.

Exemplarily, the optical axis of the sixth image acquisition device 1840 has an included angle with the predetermined plane, so that the optical axis of the sixth image acquisition device 1840 is aligned with the center of the handprint collection area. It can be understood that, in the process of practical application, due to the limitation of the layout space in the handprint collection system, the optical axis of the third image collection device 1840 has an included angle with the predetermined plane, so that it can be compared with the fourth image The collection device 1820 and the fifth image collection device 1830 are far away from the handprint collection area. In this way, there is sufficient space in the handprint collection system for installation of multiple fourth image capture devices 1820 and multiple fifth image capture devices 1830 . It is only necessary to ensure that the center of the field of view of the sixth image capture device 1840 is aligned with the center of the handprint capture area.

Exemplarily, referring to FIGS. 18 and 19 in combination, the handprint collection system may include an illumination device 1850 . The lighting device 1850 may include one or more light sources. The lighting device 1850 may be the aforementioned lighting system or a part of the aforementioned lighting system. The first plane and the second plane are located between the lighting device 1850 and the structured light projector 100 . It can be understood that the illuminating device 1850 and the structured light projector 100 are located in the edge areas on both sides of the handprint collection system. A plurality of fourth image acquisition devices 1820 and a plurality of image acquisition devices 1830 are located in the central area of the handprint acquisition system.

Exemplarily, referring to FIGS. 18 and 20 , the handprint collection system may further include a housing 1860 . A plurality of fourth image capture devices 1820 may be disposed inside the casing. It can better ensure the stability of the positions of the multiple fourth image capture devices 1820, prevent accidental touches and the like, and ensure the clarity of the captured first target image. Exemplarily, a plurality of fifth image capture devices 1830 and sixth image capture devices 1840 may be disposed in the casing 1860 . The structured light projector 100 can be arranged outside the housing 1860, so that the structured light projector 100 can project the structured light to the handprint collection area. Exemplarily, the handprint collection system may include a support base 1870 . The housing 1860 is connected to the support base 1870 , and the structured light projector 100 can be installed on the support base 1870 . In this way, the arrangement of the supporting seat 1870 can ensure the stability of the relative position between the housing 1860 and the structure projector 100 . Exemplarily, a transparent plate 270 may be provided on the housing 1860 corresponding to the handprint collection area. The setting of the light-transmitting plate 270 can play a role of dust prevention. Exemplarily, the light-transmitting plate 270 can be installed in the casing 1860 at a set height, so as to avoid as far as possible the virtual image formed by the light of the illuminating device 1850 passing through the light-transmitting plate 270 from affecting the image collection process. The lenses of multiple fourth image capture devices 1820, multiple fifth image capture devices 1830, and sixth image capture devices 1840 are arranged in a preset plane, and the virtual images formed on the light-transmitting plate 270 The positions are more concentrated, and it is easier to determine the position of the light-transmitting plate 270 to avoid the interference of the virtual image formed on the light-transmitting plate 270 . In addition, the multiple fourth image capture devices 1820 , the multiple fifth image capture devices 1830 and the sixth image capture devices 1840 are installed more centrally, which can make the installation of the light-transmitting plate 270 more convenient. Exemplarily, the transparent plate 270 may be made of glass.

Exemplarily, the handprint collection system may also be provided with a top cover. In this way, the light source of the lighting device 1850 can prevent the user's eyes from being stimulated.

According to another aspect of the present application, a handprint collection system is provided. The handprint collection system of the embodiment of the present application can be used for simultaneous collection of four-finger fingerprints, simultaneous collection of double-thumb fingerprints, or palmprint collection. Compared with the collection of single-fingerprints, different parts of multiple fingers or palmprints are more likely to have tilt angles (including pitch angles, roll angles, and yaw angles) when collecting multi-fingerprints or palmprints, resulting in partial areas The captured image is not clear because it is not within the clear imaging range of the image capture device.

The structure of the handprint collection system 200 in some embodiments will be described below with reference to FIGS. 22 to 29 . As shown in FIGS. 22 to 24 , the handprint collection system 200 has a handprint collection area 300 . The size of the handprint collection area 300 usually needs to fit the size of the four fingers of the palm. Exemplarily, the length of the handprint collection area 300 may be 110mm-130mm, and the width of the handprint collection area 300 may be 105mm-125mm. As shown in FIG. 22 , FIG. 24 to FIG. 29 , the handprint collection system 200 may include a plurality of seventh image collection devices 2220 . It can be understood that the plurality of seventh image capture devices 2220 may include two, three, four or more seventh image capture devices, which is not limited here. The plurality of seventh image acquisition devices 2220 may adopt industrial cameras or camera modules or other types of image acquisition devices, which are not limited here. In the embodiment shown in the figure, the plurality of image capture devices includes two seventh image capture devices. Regardless of the number of seventh image capture devices 2220 , the lenses 121 of a plurality of seventh image capture devices 2220 can be arranged around the centerline and face the handprint capture area 300 . It should be noted that the centerlines mentioned here and below refer to the centerlines of the plurality of seventh image capture devices 2220 . As shown in FIGS. 24 , 27 and 29 , the centerline is a straight line MN. That is to say, the lenses 121 of the plurality of seventh image acquisition devices 2220 surround the centerline MN, so that the fields of view of the plurality of seventh image acquisition devices are basically overlapped. The central line MN may be substantially parallel to the line connecting the center of the handprint collection area 300 and the center of the whole composed of a plurality of seventh image collection devices 2220 . The lenses 121 of a plurality of seventh image acquisition devices 2220 face the handprint collection area 300, like this, after the user puts the hand into the handprint collection area 300, the lenses 121 of the plurality of seventh image acquisition devices 2220 are convenient to capture the area in the handprint collection area. 300 user's handprints.

The plurality of seventh image acquisition devices 2220 respectively have object surface subspaces that can be clearly imaged within the range of depth of field from their best object front and back. That is to say, each of the seventh image acquisition devices 2220 has its own corresponding subspace of the clearly imageable object surface. The optimal object plane of the seventh image capture device 2220 generally refers to the object plane conjugate to its image plane. Typically, when the focal length of the lens of the image acquisition device is constant, the image distance of each image acquisition device is basically a constant value, so the image plane is also determined, so the optimal object plane can be determined. Depth of field range in front of and behind the best subject is still ok For clear imaging, the space within the depth of field in front of and behind the best object is the clear imaging object surface subspace. The clearly imageable object surface subspaces of the plurality of seventh image acquisition devices 2220 partially overlap. The clear imageable object surface subspaces of the plurality of seventh image acquisition devices 2220 may jointly form a clear imageable total space. In the illustrated embodiment, two seventh image acquisition devices 2220 are included. The clearly imageable object surface subspaces of the two seventh image capture devices 2220 generally overlap along the extension direction of the centerline, and the clearly imageable object plane subspaces formed by them intersect. The total space is larger than any one of the clearly imageable object surface subspaces. Of course, when there are more seventh image acquisition devices 2220, there is intersection between any two adjacent intelligible object plane subspaces along the extension direction of the center line, and the intersection of any two adjacent intelligible object plane subspaces The set is larger than any of the clear object plane subspaces, thus, the total clear imaging space of these seventh image acquisition devices can be larger than any of the clear object plane subspaces.

The handprint collection area 300 may include a total space where an object plane can be clearly imaged. When the user puts his hand into the total space of the clear imageable object plane of the handprint collection area 300, the captured handprint images can be made clear through the lenses 121 of the plurality of seventh image collection devices 2220 without contact. In some preferred embodiments, the handprint collection area 300 may be the total space where the object plane can be clearly imaged. In this way, when the user places his hand in any area of the handprint collection area 300, there is an image collection device that can collect clear handprint images. Of course, the handprint collection area 300 can also be slightly larger than the total space of the object surface that can be clearly imaged. Typically, in the case of non-contact acquisition, the user will not place the hand close to the edge of the handprint collection area 300 to avoid contact with the surrounding area. The shell of the handprint collection area 300 and the like. Taking this factor into consideration, the handprint collection area 300 may also be slightly larger than the total space of the clear imaging object plane.

The optimal object planes of the plurality of seventh image acquisition devices 2220 have different positions in the handprint acquisition area 300, and the subspaces of the clearly imageable object planes of the plurality of seventh image acquisition devices 2220 partially overlap, which can ensure Clear imaging At any position in the total space of the object plane, at least one seventh image capture device 2220 can capture a clear image of the handprint.

The handprint collection system 200 of the embodiment of the present application can collect handprints without contact. Through partial overlapping of the clearly imageable object plane subspaces of multiple seventh image capture devices 2220, and the different positions of the optimal object planes of multiple seventh image capture devices 2220 in the handprint collection area 300, the maximum expansion can be achieved. The total space that can be clearly imaged by the plurality of seventh image acquisition devices 2220 makes the acquisition range of the handprint acquisition area 300 wider. As long as the user puts the hand into the handprint collection area 300, a clear image can be formed to improve the clarity of the handprint collection to avoid that the handprint image is unavailable in subsequent applications such as identification and reduce the impact on the cooperation of the user. Require.

Exemplarily, as shown in FIGS. 22 to 24 , the handprint collection system 200 may further include a housing 1860 . The shape of the casing 1860 can be varied, which is not limited here. For example, the housing 1860 may be substantially in the shape of a cuboid. A plurality of seventh image capture devices 2220 may be located within the housing 1860 . The handprint collection system 200 may also include a cover 180 . The cover 180 may be located above or below the housing 1860 . The handprint collection area 300 can be located in the cover body 180 . The cover body 180 and the housing 1860 can be integrally formed or connected together after separate processing, which is not done here limited. The surface of the housing 1860 facing the handprint collection area 300 can transmit light. In the illustrated embodiment, the handprint collection area 300 is defined by the enclosure of the top plate 171 of the housing 1860 and the cover 180 . Thus, the top plate 171 forms a surface facing the handprint collection area 300 . A light-transmitting opening 172 may be disposed on the top plate 171 of the housing 1860 . The top plate 171 can transmit light through the light-transmitting opening 172 . Of course, part or all of the top plate 171 may be formed of a transparent material such as glass, thereby realizing light transmission. Optionally, light transmission can also be achieved by removing the top plate 171 . In this way, the casing 1860 can protect the plurality of seventh image capture devices 2220 . The cover body 180 is equivalent to forming a dedicated area for the handprint collection area 300, which is convenient for users to identify and operate. In addition, the shell 1860 and the cover 180 can also make the overall appearance of the handprint collection system 200 more beautiful, improving the experience of the user. In the case of supplementary light, the cover body 180 can avoid the glare problem of supplementary light, and can bring good experience to users.

Exemplarily, in some optional embodiments, as shown in FIGS. 22 to 23 , the handprint collection system 200 may further include a status indicator light 191 . The status indicator light 191 can be a multi-color indicator bar, and the colors include but not limited to red, green, and blue. In this way, the state of the handprint collection system 200 is prompted to the user through the color change of the state indicator light 191 .

Exemplarily, in some optional embodiments, as shown in FIGS. 22 to 23 , the handprint collection system 200 may further include a collection guide light 192 . The collection guide light 192 can be located around the handprint collection area 300 , and is used to show the range of the handprint collection area 300 to the user, prompting the user to place four fingers or two thumbs in the handprint collection area 300 . In this way, the collection guide light 192 can guide the user to place the finger to be collected in the handprint collection area 300 more quickly, which improves the collection efficiency of the handprint collection system 200 .

Exemplarily, in some optional embodiments, as shown in FIGS. 22 to 23 , the handprint collection system 200 may further include a liquid crystal display 193 . Set in this way, the liquid crystal display 193 can be used as an interaction with the user and provide some prompts to the user, such as playing and collecting animations, displaying preview images in real time, guiding posture adjustment in real time, displaying collection results, etc.; Intuitive sense of collection.

Exemplarily, the focal lengths of the lenses 121 of the plurality of seventh image capture devices 2220 may be different. With different focal lengths, even if the seventh image capture devices 2220 have the same distance to the handprint collection area 300 , the optimum object planes of multiple seventh image capture devices 2220 can be located at different positions in the handprint collection area 300 . That is to say, different focal lengths provide a basis for setting multiple seventh image capture devices 2220 at the same position. Exemplarily, when the number of the seventh image acquisition device is two, in some optional embodiments, the focal length of one seventh image acquisition device may be 8mm, and the focal length of the other seventh image acquisition device may be 6mm . In this way, it can be ensured that in each position in the handprint collection area with different distances from the front end of the lens, the field of view of the two cameras is not much different, thereby ensuring that the resolution (DPI) of the handprint collection system 200 will not vary with the distance between each position and the camera while monotonically decreasing rapidly. By making the lenses 121 of the plurality of seventh image acquisition devices 2220 have different focal lengths, it is possible to simply and easily realize the optimal object of the plurality of seventh image acquisition devices 2220. The positions of the faces in the handprint collection area 300 are different. Also, as will be mentioned later, it is easier to set up the fill light.

Exemplarily, the distances from the front ends of the lenses 121 of the plurality of seventh image capture devices 2220 to the handprint capture area 300 may be different. The azimuthal term “front end” mentioned here and below refers to the end close to the handprint collection area 300 . In the illustrated embodiment, the handprint collection area 300 is defined by the enclosure of the top plate 171 of the housing 1860 and the cover 180 . The distance from the front end of the lens 121 to the handprint collection area 300 can be understood as the distance from the handprint collection area 300 to the top plate 171 . In the illustrated embodiment, the lens 121 faces upwards to capture the handprints. In other embodiments not shown, the lens 121 can also face down or to the side to capture the handprints. In this case, the handprint collection area 300 is relatively The position of the lens 121 also needs to be adjusted accordingly. But generally, the distance from the front end of the lens 121 to the handprint collection area 300 can be understood as the distance from the front end of the lens 121 to the partition between the seventh image acquisition device 2220 and the handprint collection area 300 . By setting the distances from the front ends of the lenses 121 of the plurality of seventh image acquisition devices 2220 to the handprint acquisition area 300 to be different, even if the parameters such as the focal length, image distance, and depth of field of these seventh image acquisition devices 2220 are substantially the same, it is possible to make Their optimal object planes are located at different positions of the handprint collection area 300, so as to realize stitching of subspaces that can clearly image the object plane. Therefore, the same seventh image capture device 2220 can be selected instead of customizing different image capture devices, thereby reducing the cost of processing and storage. Moreover, since the image acquisition devices are all of the same type, assembly errors due to wrong type of image acquisition devices can also be avoided.

Exemplarily, a plurality of image acquisition devices with different focal lengths and different distances from the front end of the lens to the handprint acquisition area can be selected, which can maximize the total clear imaging space formed by their combination.

Exemplarily, the optical axes of the lenses 121 of the plurality of seventh image capture devices 2220 may be inclined toward the central line MN. The optical axis can be shown as OP and ST as shown in FIG. 29 . In some optional embodiments, the angle between the optical axes OP and ST and the central line MN may be less than or equal to 10 degrees. By tilting the optical axis of the lens 121 toward the center line MN, the center of the field of view of the plurality of seventh image capture devices 2220 can be substantially aligned with the center of the handprint capture area 300 . Usually, when the user tends to place his hand in the central area of the handprint collection area 300, the lenses 121 of the plurality of seventh image collection devices 2220 are all inclined towards the center line MN, which can ensure that the field of view of each image collection device is approximately Overlapping, the hand is located in the center of the field of view of each seventh image acquisition device 2220, thereby ensuring the imaging quality. Of course, the present application does not exclude the situation that the optical axes OP and ST are both parallel to the central line MN.

Exemplarily, the distance from the center of the field of view on the optimal object plane of the plurality of seventh image capture devices 2220 to the center line MN is less than or equal to the first preset threshold. Usually, when an existing image acquisition device captures an image, the image in the edge area far away from the center of its field of view will be deformed. Therefore, in order to ensure the imaging quality, it is expected that the seventh image acquisition device 2220 can take a picture of the hand, so as to ensure that the hand is positioned at the other side. Center of vision. By setting the distance from the center of the field of view on the optimal object plane of the plurality of seventh image acquisition devices 2220 to the centerline MN to be less than or equal to the first preset threshold, it can be ensured as much as possible When the hand is placed in the handprint collection area 300 for shooting, each image collection device can capture approximately the same hand parts, and the hand can be located in the central area of the field of view of each seventh image collection device 2220, and avoid being located in the edge area of the field of view, so Therefore, distortion of the images of the handprints collected by the plurality of seventh image acquisition devices 2220 can be avoided, and the image fidelity of the handprints collected by the plurality of seventh image acquisition devices 2220 is greatly improved. Exemplarily, the first preset threshold is 0-10 mm; preferably, the first preset threshold may be 0-5 mm; further preferably, the first preset threshold may be 0-2 mm.

Exemplarily, as shown in FIG. 22 , FIG. 25 to FIG. 27 , and FIG. 28 , the handprint collection system 200 may further include a first supplementary light 2230 . The first supplementary light 2230 may be a supplementary light provided in an integral or split structure, which is not limited here. The first fill light 2230 may be located around the plurality of seventh image capture devices 2220 . Moreover, the light-emitting surface of the first supplementary light 2230 may face the handprint collection area 300 . The first supplementary light 2230 can be generally in various shapes such as a circle, a square, a rhombus, or an ellipse, which is not specifically limited here. The first supplementary light 2230 can be controlled in any suitable manner such as automatic or manual, which will not be described here. The first supplementary light 2230 can provide auxiliary light for the surrounding environment under the condition of lack of light. The first supplementary light 2230 can also have the ability to adjust brightness, color temperature and so on. The first supplementary light 2230 is mainly for supplementary light at the bottom of the palm lines such as four fingers or two thumbs. In this way, when the light is insufficient, the first supplementary light 2230 can provide auxiliary light for the handprint collection area 300 and the surroundings of the plurality of seventh image acquisition devices 2220, so as to facilitate the acquisition of the lenses 121 of the plurality of seventh image acquisition devices 2220. A better shooting environment is provided, which is more conducive to collecting clear handprint images in any environment, and reduces the noise of the collected images caused by insufficient light; thereby improving the collection effect of the handprint collection system 200; The applicable scenarios of the handprint collection system 200 are expanded, and the versatility of the handprint collection system 200 is improved.

Exemplarily, as shown in FIG. 23 and FIG. 24 , the handprint collection system 200 may further include a light-transmitting plate 140 . The light-transmitting plate 140 can be generally in various shapes such as square, circular, or elliptical, which is not limited here. The plurality of seventh image capture devices 2220 and the handprint capture area 300 may be respectively located on both sides of the light-transmitting plate 140 . It can be known from the foregoing that the plurality of seventh image capture devices 2220 includes the lens 121 . Lens 121 usually requires careful maintenance. The plurality of seventh image acquisition devices 2220 and the handprint collection area 300 can be respectively located on both sides of the light-transmitting plate 140, and the plurality of seventh image acquisition devices 2220 can pass through the light-transmitting plate 140 to collect and collect handprints in the handprint collection area 300. shoot. Moreover, the light-transmitting plate 140 can also divide a plurality of seventh image acquisition devices 2220 and the handprint collection area 300 on its two sides, which is equivalent to forming two separate spaces. , will not touch the lenses 121 of the plurality of seventh image acquisition devices 2220, avoiding the lens 121 of the plurality of seventh image acquisition devices 2220 from being shaken or damaged by hand friction and collision, and improving the plurality of lenses 121 to a certain extent The stability of the seventh image acquisition device 2220 prolongs the service life of the lens 121 .

Exemplarily, in the case that the distances from the front ends of the lenses 121 of the plurality of seventh image acquisition devices 2220 to the handprint acquisition area 300 are the same, the centers of the front ends of the lenses 121 of the plurality of seventh image acquisition devices 2220 are located at the center line MN vertical to the lens plane. In some of the aforementioned examples Among them, the optical axes of the lenses 121 of the plurality of seventh image acquisition devices 2220 can be inclined toward the center line MN, so that the plurality of lenses 121 will also be inclined to each other, and the planes of the plurality of lenses 121 may not be coplanar, and for the convenience of description, more A plane where the center of the front end of the lens 121 of the seventh image capture device 2220 is located and perpendicular to the centerline MN is defined as a lens plane. The distance from the light emitting surface of the first fill light 2230 to the lens plane is less than or equal to the second preset threshold. It should be noted that, when the distance from the light-emitting surface of the first supplementary light 2230 to the lens plane is greater than the second preset threshold, the first supplementary light 2230 will provide a sufficient response to the lenses 121 of the plurality of seventh image capture devices 2220 in the case of insufficient light. The collection effect will be significantly reduced. In some optional embodiments, the light emitting surface of the first fill light 2230 may be coplanar with the lens plane. In this way, the first supplementary light 2230 can further provide better supplementary light effects for the lenses 121 of the plurality of seventh image capture devices 2220 and reduce lamp shadows.

Exemplarily, as shown in FIG. 22 , FIG. 25 and FIG. 26 , the first fill light 2230 may be in a ring shape surrounding a plurality of seventh image capture devices 2220 . In this way, the first supplementary light 2230 can provide supplementary light in the general circumferential direction of the plurality of seventh image capture devices 2220. Typically, the user's hand will be placed in the central area of the handprint collection area 300 for shooting The first supplementary light 2230 can be arranged in a ring shape surrounding the plurality of seventh image acquisition devices 2220 and can be arranged around the hand, thereby providing uniform supplementary light from all directions, avoiding the contrast between light and dark in the handprint image, and further improving image quality.

Exemplarily, as shown in FIG. 22 , FIG. 25 and FIG. 26 , the first supplementary light 2230 may include a first C-shaped light board 131 and a second C-shaped light board 132 . Openings of the first C-shaped lamp panel 131 and the second C-shaped lamp panel 132 are opposite and surrounded to form a ring. It can be understood that the first C-shaped lamp panel 131 and the second C-shaped lamp panel 132 may be in the same semi-circular shape, or one may be a superior arc and the other may be a inferior arc, which is not limited here. Preferably, the ring may have an inner diameter of about 80-90 mm and an outer diameter of about 100-110 mm. With this arrangement, the first fill light 2230 can be replaced relatively easily by two C-shaped lamp panels enclosing each other to form a ring-shaped first fill light 2230; if one of the C-shaped lamp panels fails, only the corresponding one needs to be replaced. The C-shaped light board is sufficient, and there is no need to replace the entire first supplementary light 2230, which reduces maintenance costs. In addition, the shape of the C-shaped light board is smoother, and there are almost no abrupt edges and corners, which is equivalent to protecting multiple seventh image acquisition devices 2220 .

Exemplarily, as shown in FIGS. 22 , 25 and 27 , the ring shape may have a notch 133 . The notch 133 can be through or a notch 133 connected with a ring, which is not limited here. As shown in FIG. 22 , FIG. 24 to FIG. 29 , the handprint collection system 200 may further include a structured light projector 100 . The structured light transmitter 100 may adopt various structured light projectors 100 known in the prior art or may appear in the future, which is not limited here. The projecting end of the structured light projector 100 can be disposed in the notch 133 . In this way, the light emitted from the projecting end of the structured light transmitter 100 can be emitted through the gap 133 . That is to say, the light emitted from the projection end of the structured light transmitter 100 passes through the ring around the plurality of seventh image capture devices 2220 . It can be known from this that the light emitted from the projection end of the structured light transmitter 100 is also located around the plurality of seventh image capture devices 2220 . Exemplarily, as shown in FIG. 25 , the handprint collection system 200 may further include a base 194 . The structured light projector 100 and the plurality of seventh image acquisition devices 2220 may both be supported on the base 194 . In this way, base 194 can support and secure A fixed structured light projector 100 and a plurality of seventh image acquisition devices 2220. With this arrangement, the structured light projector 100 can cooperate with multiple seventh image acquisition devices 2220 to form three-dimensional topography data of fingers, making the handprint information collected by the handprint collection system 200 more three-dimensional and vivid. In addition, the structured light projector 100 can cooperate with multiple seventh image acquisition devices 2220 to perform real-time posture detection of the finger, and guide the user to adjust the posture of the finger.

Exemplarily, along a lateral direction perpendicular to the central line MN, the distance between the first fill light 2230 and the plurality of seventh image capture devices 2220 is less than or equal to a third preset threshold. Exemplarily, the third preset threshold may be 0-10mm; preferably, the third preset threshold may be 0-5mm; further preferably, the third preset threshold may be 0-2mm. When the distance between the first supplementary light 2230 and the plurality of seventh image capture devices 2220 is large, the light emitted by the first supplementary light 2230 may not be able to illuminate the user's hand vertically. The light is deformed after being refracted by the transparent plate 270 between the seventh image acquisition device 2220 and the handprint collection area 300 , or the imaging is affected due to the light reflected by the transparent plate 270 .

Exemplarily, as shown in FIGS. 22 to 29 , the handprint collection system 200 may further include a second supplementary light 2260 . The second supplementary light 2260 can generally be in various shapes such as strip, circle, square, rhombus, or ellipse, which are not specifically limited here. As shown in FIG. 24 and FIG. 26 , the projection of the second fill light 2260 in the reference plane perpendicular to the central line MN may be located in the edge area and/or peripheral area of the projection of the handprint collection area 300 in the reference plane. Thus, the second supplementary light 2260 can be prevented from entering the field of vision of the plurality of seventh image capture devices 2220 and the light of the first supplementary light 2230 can be avoided. The second fill light 2260 is mainly for the side fill light of, for example, four fingers or two thumbs. The second fill light 2260 and the plurality of seventh image capture devices 2220 are located on the same side of the handprint capture area 300 . For example, it is located below the handprint collection area 300 . The second supplementary light 2260 may be located between the handprint collection area 300 and the plurality of seventh image collection devices 2220 . The second fill light 2260 can be inclined towards the center of the handprint collection area 300 . The angled second fill light 2260 can illuminate the side of the hand vertically, thus maximizing the brightness of the part being photographed.

Exemplarily, as shown in FIG. 23 and FIG. 24 , the handprint collection system 200 may further include a light-transmitting plate 270 . The light-transmitting plate 270 can be generally in various shapes such as square, circular, or oval, which is not limited here. The plurality of seventh image capture devices 2220 and the handprint capture area 300 may be respectively located on both sides of the light-transmitting plate 270 . The lenses 121 of the plurality of seventh image capture devices 2220 generally require careful maintenance. The plurality of seventh image acquisition devices 2220 and the handprint acquisition area 300 can be respectively located on both sides of the light-transmitting plate 270, and the plurality of seventh image acquisition devices 2220 can collect images of objects in the hand-print acquisition area 300 through the light-transmitting plate 270 . Moreover, the light-transmitting plate 270 can also divide a plurality of seventh image acquisition devices 2220 and the handprint collection area 300 on its two sides, which is equivalent to forming two separate spaces. , will not touch the lenses 121 of the plurality of seventh image acquisition devices 2220, avoiding the lens 121 of the plurality of seventh image acquisition devices 2220 from being shaken or damaged by hand friction and collision, and improving the plurality of lenses 121 to a certain extent The stability of the seventh image acquisition device 2220 prolongs the service life of the lens 121 .

Exemplarily, as shown in FIG. 24 , the second supplementary light 2260 may be located between the handprint collection area 300 and the light-transmitting plate 270 . In combination with the foregoing, it can be seen that the sequence between the handprint collection area 300 and the plurality of seventh image acquisition devices 2220 (including both) is the handprint collection area 300, the second supplementary light 2260, the light-transmitting plate 270, and the plurality of seventh image acquisition devices 2220. Seven image acquisition devices 2220 . In this way, the second supplementary light 2260 can provide supplementary light to the collection environment of the plurality of seventh image collection devices 2220 above the light-transmitting plate 270 . In the embodiment where the handprint collection system 200 is further equipped with the first supplementary light 2230, the second supplementary light 2260 and the first supplementary light 2230 can respectively monitor the collection environment of the plurality of seventh image collection devices 2220 at high and low positions. For supplementary light, the handprint collection system 200 can obtain a handprint image with better contrast. At this time, the effect of the second supplementary light 2260 is more significant. The second fill light 2260 is closer to the handprint collection area 300 than the light-transmitting plate 270 , which can avoid forming a shadow of the second fill light 2260 on the light-transmitting plate 270 and affecting the image collection effect.

Exemplarily, as shown in FIG. 22 , FIG. 25 and FIG. 26 , the second supplementary light 2260 may include a first lamp board 2261 , a second lamp board 2262 and a third lamp board 2263 . The projection of the first light board 2261 may be located on the fingertip side of the handprint collection area 300 . The projection of the second light board 2262 may be located on the left side of the handprint collection area 300 . The projection of the third lamp board 2263 may be located on the right side of the handprint collection area 300 . The first lamp panel 2261, the second lamp panel 2262 and the third lamp panel 2263 may form a "door"-shaped structure. The “gate”-shaped structure generally encloses the projection of the handprint collection area 300 on the reference plane. The first light board 2261 , the second light board 2262 and the third light board 2263 may be connected as one or set independently of each other, which is not limited here. It can be understood from the foregoing that the opening of the “door”-shaped structure may be roughly consistent with the direction in which the fingers enter and exit the fingerprint collection area 300 . In this way, the first light board 2261 can supplement the light on the fingertip side of the finger placed in the handprint collection area 300 . The second light board 2262 can supplement the light on the left side of the finger placed in the handprint collection area 300 . The third light board 2263 can supplement the light on the right side of the finger placed in the handprint collection area 300 . The combined effect of the three can make the second supplementary light 2260 provide supplementary light to the side of the finger, which improves the collection effect of the handprint collection system 200 .

Exemplarily, as shown in FIG. 28 , there is a first angle α between the first light board 2261 and the handprint collection area 300 . As shown in FIG. 27 , there is a second included angle β between the second light board 2262 and the handprint collection area 300 . There is a third angle γ between the third lamp board 2263 and the handprint collection area 300 . The first included angle α is smaller than the second included angle β. The second included angle β is equal to the third included angle γ. Preferably, the first included angle α may be 15-20°. Preferably, the second included angle β may be 30-40°. Usually, the included angle (obtuse angle) between the finger surface on the left and right sides of the finger and the finger surface of the finger pulp is substantially the same, so the second included angle β may be equal to the third included angle γ. Moreover, the angle (obtuse angle) between the finger surface of the fingertip and the finger surface of the finger pulp is relatively large, compared with the angle (obtuse angle) between the finger surface of the left and right sides of the finger and the finger surface of the finger pulp Smaller, the first included angle α is smaller than the second included angle β and the third included angle γ, so that the first light board 2261, the second light board 2262 and the third light board 2263 can all illuminate the corresponding side of the finger vertically, Thereby increasing the brightness of the area being photographed. With such setting, the second supplementary light 2260 can generally provide good supplementary light to the sides of the fingers in the clear imaging space, which improves the supplementary light effect; and further improves the overall collection effect of the handprint collection system 200 .

Exemplarily, the handprint collection system 200 may further include an eighth image collection device (not shown in the figure). The eighth image capture device may be located on the side of the plurality of seventh image capture devices 2220 . The optical axis of the lens 121 of the eighth image capture device can be inclined towards the centerline MN, and is used to capture images in the handprint capture area 300 . In this way, the eighth image acquisition device can better acquire the handprint information on a plane different from the plane captured by the seventh image acquisition device 2220, so as to obtain a full single-fingerprint image of each finger. Full-single-fingerprints include fingerprints on the front and two sides of the finger pad of a single finger, and can process the image of a full-single-finger fingerprint to obtain a fingerprint that simulates rolling and printing. A plurality of seventh image capture devices 2220 may be used to capture at least one of four-fingerprints, double-thumbprints, flat-palm palmprints, and side-palm palmprints. When the seventh image capture device 2220 is used in conjunction with the eighth image capture device, all single finger prints can be photographed. The processing device may stitch the images collected by the seventh image collection device 2220 and the eighth image collection device onto the 3D fingerprint surface of the 3D fingerprint model to obtain a 3D fingerprint image. Moreover, with the cooperation of the eighth image acquisition device and the structured light projector, the area of the fingerprint obtained is much larger than that obtained by traditional contact, and the acquisition quality is higher. When a high-quality, full single-fingerprint is required, you only need to place the finger in the image acquisition area and hover for a moment to obtain a high-definition and complete single-fingerprint image. There is no need to roll a finger from side to side on the capture device to increase the total contact area. This solution is especially suitable for individuals who are not compliant, and there will be no blurring of fingerprint images. Of course, the eighth image capture device may also be located on the side of the fingertips of the plurality of seventh image capture devices 2220, and the user captures the fingerprints at the fingertips. The eighth image capture device can also be located on the root side of the plurality of seventh image capture devices 2220, and is used to capture fingerprints at the heel of the finger (that is, the part above the first knuckle, closest to the first knuckle line) . Or simultaneously include any combination of the above-mentioned eighth image acquisition device. It is not intended herein to limit the number of the eighth image acquisition device.

The handprint collection system includes multiple seventh image collection devices 2220 and eighth image collection devices. , flat palm and side palm and other collection requirements make the application of the handprint collection system wider.

Exemplarily, there may be multiple groups of eighth image acquisition devices. Each group corresponds to a seventh image acquisition device 2220 . That is to say, the number of groups of the eighth image capture device is the same as the number of the seventh image capture device 2220 , and there is a one-to-one correspondence. Each group of eighth image acquisition devices may be used to capture images in the clearly imageable subspaces of the corresponding seventh image acquisition devices. The eighth image capture device group and the seventh image capture device corresponding to each other have matching sharply imageable subspaces. Exemplarily, each group may include two eighth image capture devices, which are respectively located on the left and right sides of the plurality of seventh image capture devices 2220, and are used to cooperate with the corresponding seventh image capture devices 2220 to capture full single-finger images. . Of course, each group may also include one eighth image acquisition device or more eighth image acquisition devices. In this way, when the user's hand is located in a subspace that can be clearly imaged by a seventh image capture device 2220, a group of eighth image capture devices corresponding to the seventh image capture device 2220 can simultaneously capture images of the user's hand, thereby ensuring the The seventh image acquisition device 2220 and a corresponding group of eighth image acquisition devices can clearly image, thereby ensuring that the front and side handprints can be taken photo effects.

Exemplarily, there are N seventh image acquisition devices 2220 . The eighth image acquisition device is a group. The handprint collection system 200 may also include an angle adjustment device (not shown in the figure). The angle adjustment device may be connected to the eighth image capture device for making the optical axis of the lens 121 of the eighth image capture device have at least N tilt angles with respect to the central line MN. At the n-th inclination angle, n belongs to {1, 2, 3, . . . , N-1, N}. The eighth image acquisition device is used to capture the clear imaging subspace of the nth seventh image acquisition device. Describe with a specific embodiment, assume that the seventh image acquisition device is 3, that is, N is 3; The eighth image acquisition device is a group; The angle adjustment device can make the optical axis of the lens of the eighth image acquisition device relatively There can be at least 3 inclination angles on the centerline MN; when located at the first inclination angle, the eighth image acquisition device can be used to shoot the clear imaging subspace of the first seventh image acquisition device; When tilted at an angle, the eighth image capture device can be used to shoot the clear imaging subspace of the second and seventh image capture devices; at the third tilt angle, the eighth image capture device can be used to shoot the third Seven subspaces that can be clearly imaged by the image acquisition device. With such an arrangement, through the angle adjustment device and a group of eighth image acquisition devices, clear handprint images can be taken with all the seventh image acquisition devices 2220, which reduces the configuration of the eighth image acquisition device and reduces the number of parts , reduces the cost; also reduces the space it occupies, optimizes the structure of the handprint collection system 200, and facilitates its miniaturization design.

The handprint collection system of the embodiment of the present application can be used for simultaneous collection of four-finger fingerprints, simultaneous collection of double-thumb fingerprints, or palmprint collection. Compared with the collection of single-fingerprints, different parts of multiple fingers or palmprints are more likely to have tilt angles (including pitch angles, roll angles, and yaw angles) when collecting multi-fingerprints or palmprints, resulting in partial areas The captured image is not clear because it is not within the clear imaging range of the image capture device. In the prior art, the user is required to place the hand at a specific height, and there are many restrictions on the posture of the hand, so that the handprint can be clearly imaged. In order to at least partly solve the above technical problems, an embodiment of the present application provides a non-contact method for collecting handprints of a target object. The device for realizing the non-contact target object's handprint collection method may be the above-mentioned handprint collection system 200 . In one or some embodiments, the device implementing the non-contact target object handprint collection method may be a handprint collection system 200 as shown in FIGS. 22-29 . The handprint acquisition system may include a preview image acquisition device, one or more preset image acquisition devices (for example, multiple image acquisition devices that can clearly image the overlapping subspaces of the object surface), wherein the preview image acquisition device may be one or more One of the preset image capture devices may also be a separate image capture device independent of one or more preset image capture devices. The preview image capture device and/or the preset image capture device may be any type of image capture device, such as an industrial camera, a camera module, and the like. Fig. 30 shows a schematic flowchart of a method 3000 for collecting non-contact handprints of a target object according to an embodiment of the present application. Step S3030 and processing step S3040.

Acquisition step S3010, using the preview image acquisition device to acquire a preview handprint image for the handprint collection area, the handprint collection area is used to place hands, and the preview image includes the handprint of the target object, The target object includes at least one of the four fingers of the hand, the two thumbs, and the palm. The aforementioned three-dimensional target may be or include a target object. One or more image capture devices in the handprint capture system 200 may include preview image capture devices, and target images captured by the one or more image capture devices may include preview handprint images.

When collecting the handprint image of the target object, the target object can be put into the handprint collection area of the handprint collection system 200 at one time. For example, when the target object is four fingers, the four fingers can be put into the handprint collection area at the same time, instead of a single finger being put in four times. In this way, the fingerprint images of the four fingers can be obtained at one time, which improves the collection efficiency and reduces the User cooperation times. Referring back to FIG. 25, the preset image capture device may be the seventh image capture device 2220, and the preview image capture device may be one of the preset image capture devices 2220, or other separate image capture devices (such separate image capture devices are not shown in FIG. 25 ). image capture device). In one embodiment, the preview image of the handprint of the target object can be collected by a preview image collection device. When the user puts the hand in the handprint collection area, the preview image collection device can obtain the user's preview handprint image at the current collection time (for example, time t).

Attitude judging step S3020, determine whether the posture of the posture judging object is qualified based on the previewed handprint image, if qualified, go to the acquisition step S3030; wherein the posture includes at least one of the following posture indicators: position, yaw angle, roll angle and the pitch angle; wherein, the attitude judgment object is the target object or the processing object.

Exemplarily, according to the acquired preview palmprint image, it can be judged whether the posture of the gesture judgment object at the preview palmprint image acquisition time (for example, the above-mentioned time t) is qualified. In one embodiment, the gesture judgment object may be a target object or a processing object. A processing object can represent a single finger or a palm contained in a target object. For example, in the case that the target object is four fingers of the hand, the posture judgment object may be the target object (four fingers) or the processing object (single finger). When the posture judgment object is four fingers, the four fingers as a whole can conclude that the posture is qualified or unqualified. When the posture judgment object is a single finger, a conclusion of qualified or unqualified posture will be given for the single finger. Similarly, in the case that the target object is the double thumbs of the hand, the posture judgment object may be the target object (double thumbs) or the processing object (single thumb). In the case where the target object is the palm, the gesture judgment object can only be the palm.

In one example, the pose determination object may be a target object. Judging whether the posture of the target object is qualified or not may include judging whether one or more of the position, yaw angle, roll angle, and pitch angle of the target object in the previewed handprint image are qualified or not. In this case, the posture of the target object is considered qualified only if the overall posture of the target object (for example, all four fingers) is qualified, and the subsequent acquisition step S3030 can be performed. If the posture of any one or more processing objects in the target object is unqualified, it may be considered that the posture of the target object is unqualified, and the subsequent acquiring step S3030 is not performed. The above position may indicate whether the position of the target object in the previewed palmprint image is to the left or to the right, whether it is in the central area of the previewed palmprint image, and so on.

In another example, the gesture judgment object may be a processing object. Judging the posture of the processing object Whether the status is qualified may include judging whether one or more of the position, yaw angle, roll angle, and pitch angle of the processing object in the previewed palmprint image is qualified. In this case, if the poses of the currently processed objects are all qualified, the subsequent acquiring step S3030 can be performed.

The acquisition step S3030 is to acquire the to-be-processed palmprint image corresponding to the processing object included in the target object; the to-be-processed handprint image includes the processing object with a qualified posture, and the processing object is a single finger or palm in the target object; the to-be-processed handprint image It is determined according to the target handprint image corresponding to the processing object; the target handprint image is collected by the target image acquisition device for the handprint collection area within the specified time interval from the preview handprint image collection time; the target handprint image includes the target The handprint of the object; the clearly imageable object surface subspace of the target image acquisition device matches the height of the object to be processed. One or more image acquisition devices in the handprint collection system 200 may include target image capture devices, and target images captured by the one or more image capture devices may include target handprint images.

After it is determined that the posture of the processing object is qualified, the to-be-processed handprint image corresponding to the processing object is acquired. If the postures of all processing objects corresponding to the target object are qualified, the to-be-processed handprint images corresponding to all the processing objects can be obtained; if the postures of some processing objects corresponding to the target object are qualified, the to-be-processed handprint images corresponding to this part of the processing objects can be obtained.

The handprint image to be processed is determined according to the target handprint image. The target handprint image includes the handprint of the target object, the to-be-processed handprint image only includes a single processing object, and the to-be-processed handprint image is determined according to the target handprint image. For example, when the target object is four fingers and the processing object is the middle finger, the target handprint image includes four fingers, and the to-be-processed handprint image is an image that includes only the middle finger and is intercepted from the target handprint image including four fingers. When the processing object is a single finger, the palmprint image to be processed may be an image of the entire finger of the single finger, an image of the fingerprint region of the single finger, or an image of the fingerprint of the single finger, which is not limited here. When the target handprint image includes a structured light channel and an unstructured light channel, the handprint image to be processed can be determined by performing target segmentation on the unstructured light channel of the target handprint image.

It can be understood that the step of determining the handprint image to be processed according to the target handprint image may be performed during the execution of the acquisition step S3030, or during the execution of the posture judgment step S3020 or in the posture judgment step S3020 It is performed after and before the execution of the acquiring step S3030.

The to-be-processed handprint image of the processing object is determined according to the target handprint image corresponding to the processing object. It can be understood that different processing objects belonging to the same target object may correspond to the same target handprint image, or may correspond to different target The handprint images may correspond to the same target image acquisition device, or may correspond to different target image acquisition devices. If at a certain moment, according to the heights of the first processing object and the second processing object belonging to the same target object, it is determined that the target image acquisition devices corresponding to the two are the same, then the handprint image of the same target collected by the target acquisition device can be Two to-be-processed handprint images containing the two processing objects are acquired respectively. If at a certain moment, according to the heights of the first processing object and the second processing object belonging to the same target object, It is determined that the first processing object and the second processing object correspond to different target image acquisition devices. For any two different processing objects, it is also possible to acquire two images containing the two processing objects according to two different target handprint images. Handprint image to be processed.

The target handprint image of the processing object is collected by the target image acquisition device, and the clearly imageable object surface subspace of the target image acquisition device matches the height of the processing object. The height of the processed object is used to determine the target image capture device. The height of the processing object may be determined according to a sensor such as a distance sensor, or may be determined according to a preview image of the handprint. The image acquisition device of the handprint acquisition system has various setting schemes.

Solution 1, the image acquisition device of the handprint acquisition system only includes a preview image acquisition device, and the preview image acquisition device is not only used for preview, but also for acquisition of target handprint images. At this time, the preview image acquisition device has a large enough clear imageable object plane space, which is sufficient for clear imaging of four fingers, double thumbs or palms. The target image acquisition device is the preview image acquisition device, and the clearly imageable object surface subspace of the preview image acquisition device matches the height of the processing object.

Under this scheme, the target handprint image can be a preview image of the handprint image that is judged to be qualified in the attitude of the processing object, or it can be an image re-taken after the attitude of the processing object is determined to be qualified (when the preview image is at a lower resolution This is usually done when a shot is taken, or when a reshoot is thought to allow for better focus).

Option 2, including a preview image acquisition device and a preset image acquisition device different from the preview image acquisition device (the preset image acquisition device can be one or more), and the preview handprint image collected by the preview image acquisition device is only used for judging For hand gestures, the preset image capture device is used to capture target handprint images, that is to say, the target image capture device is selected from the preset image capture devices. When there are multiple preset image acquisition devices, the clear imageable object surface subspaces of multiple preset image acquisition devices partially overlap. It is determined by the matching between the subspace of the object surface and the height of the processing object. For example, there are preset image acquisition devices A and B, and the clear imageable object surface subspaces of the two are [h1h2] and [h2h3] respectively (just take the single-point coincidence as an example, the two clear imageable object surface subspaces can of course be interval coincide). When the height of the processing object falls between h1-h2, use A as the target image acquisition device; when the height of the processing object falls between h2-h3, use B as the target image acquisition device.

Under this scheme, the target handprint image can be taken almost simultaneously with the preview image After processing the height of the object, the target image acquisition device can be selected from the preset image acquisition devices, and the image collected by the target image acquisition device will become the target handprint image), or it can be determined that the attitude of the processing object is qualified, according to the preview of the handprint After determining the height of the processing object and selecting the target image acquisition device according to the height, the image is re-taken by the target image acquisition device.

Scheme 3, including multiple preset image acquisition devices, the preview image acquisition device belongs to the preset image As one of the image acquisition devices, the preview image acquisition device collects the preview handprint image for judging the hand posture, the preview image acquisition device and the preset image acquisition device are both used for the acquisition of the target handprint image, which image acquisition device is used specifically Acquisition as the target image acquisition device should be determined according to the matching between the clearly imageable object surface subspace of the image acquisition device and the height of the processing object. For example, there is preview image acquisition device C, preset image acquisition devices A and B, and the clear imageable object surface subspaces of the three are h0-h1, h1-h2 and h2-h3 respectively. When the processing object height falls between h0-h1, use C as the target image acquisition device; when the processing object height falls between h1-h2, use A as the target image acquisition device; when the processing object height falls within h2- Between h3, use B as the target image acquisition device. That is to say, the target image capture device may be a preview image capture device, or a preset image capture device. Under this scheme, the target handprint image can be a preview handprint image (in this case, after the height of the processing object is determined according to the preview handprint image, the preview image acquisition device is selected as the target image acquisition device), or it can be It is shot almost simultaneously with the preview handprint image (in this case, when the preview image acquisition device collects images, the preset image acquisition device is also collecting images, after determining the height of the processing object according to the preview handprint image, the target selected according to the height The image acquisition device is a preset image acquisition device rather than a preview image acquisition device, and the image collected by the target image acquisition device also becomes the target handprint image), it can also be determined after the attitude of the processing object is qualified, and the processing object is determined according to the preview handprint image and the image re-taken by the target image capture device after selecting the target image capture device according to the height (the target image capture device can be a preview image capture device or a preset image capture device at this time).

It is understandable that no matter whether the target image acquisition device is a preview image acquisition device or not, the target acquisition device corresponding to the processing object can meet the following requirements: the clearly imageable object surface subspace of the target acquisition device corresponding to the processing object is consistent with the height of the processing object. match. In this way, a clearly imaged processing object can be obtained in the target palmprint image. In one embodiment, the height of the target object is determined as a whole, and the height of the target object is used as the height of each processing object included in the target object. At this time, all processing objects can correspond to the same target image acquisition device. This solution is suitable for posture judgment Handles determination of object height when the object is a full hand. In another embodiment, the height can be determined individually for each processing object. In the case that two processing objects have different heights, the two processing objects may correspond to different target image acquisition devices, that is, correspond to different target handprint images.

The target handprint image is collected by the target image collection device for the handprint collection area within a specified time interval from the preview handprint image collection time. The designated time interval may be set to any appropriate duration as required. For example, the specified time interval may be [0, Δt], where the magnitude of Δt may be in the range of [1 100] ms, for example. The collection time of the target handprint image should not be too long from the collection time of the preview handprint image, so as to avoid a large change in the posture of the human hand. Although the posture of the object is judged to be qualified according to the preview handprint image, the posture of the actual collection is unqualified. Case. The target handprint image is collected within a specified time interval from the preview handprint image collection time, which can include the situation that the target handprint image and the preview handprint image are collected almost at the same time, and can also include the target handprint image collected after the preview handprint image is collected Condition. In particular, the above objectives The handprint image is the case of previewing the handprint image, and it can also be regarded as the case of collecting the target handprint image within a specified time interval from the collection time of the previewed handprint image.

Processing step S3040, processing the acquired handprint image to be processed.

For example, processing the acquired handprint image to be processed may include one or more of the following: performing fingerprint recognition on the handprint image to be processed; transforming the handprint image to be processed into a simulated stamped fingerprint image. The transformation of the to-be-processed handprint image into a simulated stamped handprint image can be realized by performing three-dimensional reconstruction and planar expansion of the to-be-processed handprint image, which will be described below.

According to the non-contact target object handprint collection method and the handprint collection system of the embodiment of the present application, the preview handprint image collected by the preview image collection device is used to judge the posture of the posture judgment object. After the posture is qualified, according to the processing object The handprint image of the object to be processed is obtained by matching the target handprint image captured by the target image acquisition device. This solution guarantees fast posture judgment by previewing the image acquisition device, and determines the target image acquisition device that matches its height according to the height of the processing object, so that the handprint acquisition device in the embodiment of the present application can be used in a larger space and in a more relaxed manner. Handprints are clearly imaged under the limitation of hand posture, thereby reducing the requirements for user cooperation and improving user experience.

Exemplarily, the target image acquisition device is selected from a plurality of image acquisition devices that can clearly image partly overlapping subspaces of the object surface, and the method may further include: determining the height of the processing object based on the previewed handprint image; In the acquisition device, select an image acquisition device that can clearly image the subspace of the object surface and match the height as the target image acquisition device corresponding to the processing object; the acquisition step S3030 may include: using the target image acquisition device to preview the handprint image at a distance from the handprint collection area Collect target handprint images in the specified time interval of collection time, and determine the handprint images to be processed according to the target handprint images; The target handprint image determines the to-be-processed handprint image; or, the to-be-processed handprint image is acquired; wherein, the to-be-processed handprint image is determined according to the previewed handprint image, and the previewed handprint image is the target handprint image.

In this embodiment, the setting scheme of the image acquisition device of the handprint collection system includes scheme 2 and scheme 3 in step 3030 .

In this embodiment, the handprint acquisition system may include a plurality of image acquisition devices that can clearly image partly overlapped subspaces of the object surface (that is, the above-mentioned preset image acquisition devices), and the target image acquisition device of the processing object may be selected from among them according to the height of the processing object. out. For example, referring to the two seventh image acquisition devices 2220 in the embodiment in FIG. 25 , the subspaces of the two image acquisition devices that can be clearly imaged are partially overlapped. The height of the processing object can be determined based on the previewed handprint image (the height of each processing object can be calculated separately, or the height of the target object can be calculated as the height of the processing objects included in the target object). For example, the preview palmprint image may be collected under the illumination of a structured light source, and the structured light image corresponding to the preview palmprint image may be determined according to the preview palmprint image. Exemplarily, the structured light source is a monochromatic light source such as red light, and the red channel in the preview palmprint image is a structured light channel, which can be used as a structured light image corresponding to the preview palmprint image. According to the corresponding knot of the preview handprint image The light composition image (for example, the structured light channel of the previewed handprint image) can perform three-dimensional reconstruction on the processing object in the previewed handprint image. Based on the 3D reconstruction result, the height of the processed object can be obtained (when the connecting line between the target object and the image acquisition device is basically in the vertical direction).

The heights of the processing objects included in the target object may be the same or different. The heights of the processing objects included in the target object may be the same. For example, in Case 1, the heights of each processing object included in the target object may be calculated and the average value of some or all of the heights may be used as the final height of each processing object. For example, the respective heights of the four fingers can be calculated, and the heights corresponding to the largest and smallest heights can be selected and the average value of the two can be calculated as the height of the target object and also as the height of each processing object. The processing objects included in the target object have the same height. For another example in case 2, the height of the target object may be calculated and used as the height of the processing object. The processing objects included in the target object have different heights. For example, in case 3, the height of each processing object may be calculated separately as the height of each processing object. It can be understood that if in the posture judgment step S3020, the granularity of the posture judgment is the entire target object (that is, the posture judgment object is the target object, and the posture judgment result is whether the posture of the target object is qualified or unqualified), then when calculating the height, The heights of the processing objects included in the target object can be calculated individually or as a whole, and the heights of the processing objects can be the same or different. When the heights of the processing objects are the same, the target collection devices corresponding to the processing objects must be the same; when the processing objects have their own heights, the target collection devices corresponding to the processing objects may be different. If in the posture judgment step S3020, the granularity of posture judgment is a single processing object (that is, the posture judgment object is a single processing object, and the posture judgment result is that the posture of the processing object is qualified or unqualified), then when the height is calculated, usually each The height of the processing object is calculated separately, because at this time it is not possible to determine that the attitude of all processing objects is qualified, and only the height of the processing object with a qualified attitude can be calculated. At this time, the target acquisition device corresponding to the different processing objects included in the target object can be different.

The following description will be made by taking the preset image acquisition device including the image acquisition device C1 and the image acquisition device C2 as an example.

In one embodiment, the posture determination object is a target object including four fingers, and the height calculation is performed on the four fingers as a whole. According to the preview handprint image collected by the preview image collection device (C1 or C2 or other image collection device), it is determined that the posture of the target object is qualified. After that, determine the overall height of the four fingers according to the corresponding structured light image in the preview palmprint image (for example, calculate the heights of the index finger, middle finger, ring finger, and little finger respectively and take the average as the height of each processing object (single finger)). Next, the target collection device C1 is selected according to the height of the processing object. Since each processing object corresponds to the same height, the target collection device corresponding to each processing object is C1. Afterwards, at time T1 (within the specified time interval from T1 to the preview image collection time), the target handprint image is collected by the target image collection device C1, and finger segmentation is performed on the target handprint image to obtain the corresponding fingerprint of each processing object (single finger). The handprint image to be processed. The processing objects and their corresponding target handprint images are shown in Table 1.

Table 1

In another embodiment, the gesture determination object is a target object including four fingers, and the height calculation is performed on each finger in the target object. According to the preview handprint image collected by the preview image collection device (C1 or C2 or other image collection device), it is determined that the posture of the target object is qualified. Afterwards, the heights of the four fingers are respectively determined according to the corresponding structured light images in the previewed palmprint image (for example, the heights of the index finger, middle finger, ring finger, and little finger are respectively calculated). Next, according to the height of the object to be processed, select the corresponding target acquisition devices (for example, the target acquisition device corresponding to the index finger and little finger is C1, and the target acquisition device corresponding to the middle finger and ring finger is C2). Afterwards, at T1 moment (T1 is within the specified time interval from the preview image acquisition time), the handprint image is collected by the target image acquisition device C1, as the target handprint image of the index finger and the little finger, and the handprint image is collected by the target image acquisition device C2, The target palmprint images of the middle finger and the ring finger are used, and finger segmentation is performed on the target palmprint images of each processing object to obtain the to-be-processed palmprint image corresponding to the processing object (single finger). The processing objects and their corresponding target handprint images are shown in Table 2.

Table 2

In yet another embodiment, the posture judgment object may be each of the four fingers, and the height calculation is performed on each finger. While the preview image acquisition device (for example, C1 ) is acquiring the preview image of the handprint, another image acquisition device C2 is also acquiring images. According to the preview handprint image collected by the preview image collection device, it is determined that the posture of the processing object is qualified (for example, in the preview image collected by T1, the index finger is judged to be qualified, and other fingers are not qualified; in the preview image collected by T2, the middle finger and ring finger are judged to be qualified, and the little finger is not qualified. Qualified; in the preview image collected by T3, the little finger is judged to be qualified). After that, determine the heights of the four fingers respectively according to the corresponding structured light images in the qualified preview handprint image of the processing object (for example, calculate the height of the index finger from the preview handprint image collected by T1; calculate the height of the index finger from the preview handprint image collected by T2; , calculate the height of the middle finger and ring finger; calculate the height of the little finger from the preview handprint image collected by T2). Next, select the target acquisition devices corresponding to them according to the height of the processing object (for example, the target acquisition device corresponding to the index finger, ring finger and little finger is C1, and the target acquisition device corresponding to the middle finger is C2). Afterwards, the preview handprint image taken at T1 time C1 is used as the target fingerprint image of the index finger, and the handprint image taken at T2 time C2 is taken as the target fingerprint image of the middle finger, The preview palmprint image captured at T2 time C1 is used as the target palmprint image of the ring finger, and the preview palmprint image captured at T3 time C1 is used as the target palmprint image of the little finger. The processing objects and their corresponding target handprint images are shown in Table 3.

table 3

Correspondingly, the obtaining step S3030 may include the following steps.

In one embodiment, if after the posture judging step S3020, it is determined that the posture of the target object at the preview handprint image acquisition time is qualified based on the preview handprint image, then step S3030 includes using the target image acquisition device to measure the distance between the current preview handprint image Within the specified time interval of the collection time, image collection is performed on the target object in the handprint collection area to obtain a target handprint image. Subsequently, the handprint image to be processed is determined according to the collected target handprint image. For example, the preview image acquisition device is C1, and the image acquisition devices that can clearly image overlapping subspaces are C1 and C2, and the target image acquisition device that selects the processing object according to the height of the processing object is C2, which is equivalent to using a different preview image acquisition device. The new image acquisition device re-acquires the target handprint image. For another example, the preview image acquisition device is C1, and the image acquisition devices that can clearly image subspace overlapping are C1 and C2, and the target image acquisition device that selects the processing object according to the height of the processing object is C1. At this time, the preview image acquisition device and the target image acquisition Although the devices are the same, the resolution of the preview handprint image collected during preview may not be sufficient, so you can choose to re-acquire the target handprint image. For another example, the preview image acquisition device is C1, and the image acquisition devices that can clearly image subspace overlapping are C2 and C3, and the target image acquisition device that selects the processing object according to the height of the processing object is C3. The new image acquisition device of the acquisition device re-acquires the target handprint image.

In another embodiment, step S3030 includes acquiring a target handprint image collected by the target image collection device at the same time as the preview fingerprint image in the handprint collection area, and determining the handprint image to be processed according to the target handprint image. That is to say, when the preview image acquisition device acquires the preview handprint image for the handprint acquisition area, other image acquisition devices are used to simultaneously acquire the target handprint image for the handprint acquisition area. For example, the preview image acquisition device is C1, and the image acquisition devices that can clearly image overlapping subspaces are C1 and C2. When C1 collects the preview handprint image, C2 also uses C2 to collect the handprint image, and determines the processing object according to the preview handprint image. The target image acquisition device is C2. At this time, there is no need to use C2 to re-acquire the target handprint image, because the handprint image collected by C2 can be used as the target handprint image when C1 collects and previews the handprint image. Although the target handprint image has been collected previously, the handprint image to be processed has not been determined (for example, by segmenting the target handprint image), so it can be acquired In step S3030, the handprint image to be processed is determined according to the target handprint image.

In yet another embodiment, step S3030 includes directly acquiring the handprint image to be processed, wherein the handprint image to be processed is determined according to the previewed handprint image. At this time, the attitude of the processing object is judged to be qualified according to the preview palmprint image, and the target palmprint image of the processing object is judged to be the preview palmprint image according to the height of the processing object, and the resolution of the preview palmprint image is sufficient, so there is no need to re-acquire a new one. image as the target handprint image. That is, the target handprint image is the preview handprint image. Since the preview handprint object has been segmented for gesture judgment during posture judgment, the single-finger image segmented during posture judgment can be directly obtained as the processing object. Processing handprint images.

According to the above technical solution, the handprint images to be processed can be obtained in different ways to meet the needs of different application scenarios. Therefore, this solution has high flexibility and can balance image quality and time delay, as well as the need for user cooperation. Require.

Exemplarily, the preview image acquisition device is an image acquisition device with the shortest focal length in the image acquisition device set composed of the preview image acquisition device and a plurality of image acquisition devices.

Usually, the image acquisition device with the largest field of view/depth of field is selected as the preview image acquisition device.

In the case that the preview image capture device is one of multiple image capture devices, the set of image capture devices is a plurality of image capture devices. For example, if the preview image acquisition device is image acquisition device C1 and the target image acquisition devices are image acquisition devices C1 and C2, then the set of image acquisition devices may include image acquisition devices C1 and C2. In a case where the preview image capture device is not one of the plurality of image capture devices, the set of image capture devices includes the preview image capture device and the plurality of image capture devices. For example, if the preview image capture device is image capture device C3, and the target image capture devices are image capture devices C1 and C2, then the set of image capture devices may include image capture devices C1, C2 and C3.

As an example and not a limitation, the lenses of the image capture devices in the image capture device set may be set at the same height, but their corresponding focal lengths are different. For example, the focal lengths of the two image capture devices C1 and C2 may be 6 millimeters (mm) and 8 mm, respectively. The preview image acquisition device may be an image acquisition device C1 with a focal length of 6 mm. With such an arrangement, based on the image acquisition device with a short focal length, a preview handprint image with a larger field of view can be collected for the target object, so as to collect the complete target object as much as possible.

Exemplarily, the target handprint image is collected under the illumination of a structured light source and an unstructured light source, and the handprint image to be processed includes a structured light channel and an unstructured light channel, and the processing steps may include: for each processing object, according to The structured light channel of the handprint image to be processed corresponding to the processing object determines the three-dimensional information of the processing object, and performs expansion transformation on the unstructured light channel of the handprint image to be processed corresponding to the processing object according to the three-dimensional information, to obtain the corresponding The expanded image; according to the expanded image, the simulated imprint image corresponding to the processing object is obtained.

In one embodiment, the target handprint image can be collected simultaneously under the illumination of a structured light source and an unstructured light source (such as a visible light source), and at this time, the target handprint image corresponds to the structured light image and the unstructured light image. Exemplarily, structured light sources and unstructured light sources (for example, lighting When the bright light source) is of different colors, the target handprint image can have a structured light channel and an unstructured light channel (which can be respectively used as a structured light image and an unstructured light image corresponding to the target handprint image), and the corresponding handprint image to be processed also It can include structured light channels and unstructured light channels. According to the structured light channel of the to-be-processed handprint image corresponding to the processing object, three-dimensional reconstruction can be performed on the processing object to determine the three-dimensional information of the processing object. The three-dimensional information may be the three-dimensional world coordinates of each point on the processing object in the world coordinate system. Exemplarily, when the structured light is striped light, the conversion relationship between the world coordinate system and the image coordinate system can be determined based on each strip under the structured light channel of the handprint image to be processed, and based on the conversion relationship The two-dimensional image coordinates of each point on the processing object are converted into three-dimensional world coordinates to obtain the three-dimensional information of the processing object. Those skilled in the art can understand the implementation of the three-dimensional reconstruction, which will not be described in detail herein.

According to the three-dimensional information, the unstructured light channel of the to-be-processed handprint image corresponding to the processing object can be expanded and transformed to obtain the expanded image corresponding to the processed object. Further processing (such as image enhancement) on the expanded image can obtain the corresponding simulated imprinted image.

According to the above technical solution, the three-dimensional information of the processing object is obtained through the structured light channel of the handprint image to be processed corresponding to the processing object, and the unstructured light channel of the handprint image to be processed corresponding to the processing object is expanded and transformed based on the three-dimensional information, According to the expanded image, the corresponding simulated imprinted image is obtained. The non-contact collected simulated imprinting image obtained by the method is more in line with the handprint collected by the user's imprinted, and higher comparison accuracy can be obtained when the simulated imprinted image and the imprinted fingerprint image are used for fingerprint comparison.

Exemplarily, determining whether the gesture of the gesture judging object is qualified based on the previewed palmprint image may include: determining a first gesture index of the processing object contained in the gesture judging object based on the previewed palmprint image, where the first posture index is selected from the posture index.

In an embodiment, the first posture index may represent an index that needs to be judged separately for each processing object (that is, judgment is performed at a granularity of a single processing object), and it is selected from posture indexes. For example, the posture judgment object is a processing object (such as a single finger), and all posture indicators of each processing object can be judged separately. For example, if the attitude judgment object is a target object (such as four fingers as a whole), the pitch angle of each finger can be judged separately, and the roll angle can be judged by four fingers as a whole. At this time, the first attitude index includes the pitch angle and does not include the roll angle . Whether the posture of the posture judging object is qualified can be determined by judging whether the first posture index of the processing object included in the posture judging object is qualified. In the case that the attitude judgment object is the target object, when judging whether the attitude of the attitude judgment object is qualified according to the attitude index, some attitude indexes can be calculated as a whole. Qualified; some attitude indicators can be calculated by a single processing object, and the attitude of the target object is determined to be qualified only when the attitude indicators of all the processing objects are qualified. Optionally, for some attitude indicators, the attitude indicator of a single processing object may be used as a representative of the overall attitude indicator of the attitude judgment object.

According to the above technical solution, it is possible to separately determine the first posture index of each processing object included in the posture judgment object, and then judge whether the posture of the posture judgment object is qualified or not according to the first posture index. This solution is flexible and helps to ensure the accuracy of attitude judgment results.

Exemplarily, determining the first gesture index of the processing object included in the gesture judgment object based on the previewed handprint image may include at least one of the following: repeating the first structured light corresponding to the processing object in the structured light channel of the previewed handprint image The unit and the second structured light repeating unit determine the pitch angle or roll angle of the object to be processed; the preview handprint image is collected under the illumination of a structured light source, and the preview handprint image can include a structured light channel; according to the preview handprint image structure The height of the two strip points of the third structured light repeating unit corresponding to the processing object in the optical channel determines the roll angle or pitch angle of the processing object; the preview handprint image is collected under the illumination of a structured light source, and the preview handprint image The fingerprint image includes a structured light channel; according to the position and/or shape of the processing object in the unstructured light channel of the preview handprint image, determine the position and/or yaw angle of the processing object; the preview palmprint image is illuminated by an unstructured light source Under acquisition, the preview handprint image includes an unstructured light channel.

The preview handprint image may be collected under the illumination of a structured light source, so the preview handprint image may include a structured light channel. In one embodiment, the structured light source can be strip light, the first structured light repeating unit in this embodiment, the second structured light repeating unit, the third structured light repeating unit and the fourth structured light in the following embodiments Each of the repeating unit, the fifth structured light repeating unit and the sixth structured light repeating unit may represent any strip in the strip light. In the preview image of the handprint, if the first structured light repeating unit and the second structured light repeating unit are arranged along a direction perpendicular to the axis of any processing object (such as a single finger), they can be used to determine the pitch angle of the processing object. For example, the height difference between the first structured light repeating unit and the second structured light repeating unit corresponding to the processing object may be determined, and the height difference is used to represent the elevation angle of the processing object. Exemplarily, if the height difference is smaller than the first height threshold, it may be determined that the elevation angle of the processing object is qualified; otherwise, it is determined that the elevation angle of the processing object is unqualified. The first height threshold can be set to any suitable value as required. Exemplarily, FIG. 31A shows a schematic diagram of a structured light channel image according to an embodiment of the present application. It should be noted that the structured light channel image shown in Figure 31A is generated by irradiating structured light to the processing object (single finger). Both band and scatter repeat units are formed. The structured light repeating unit described herein mainly refers to each strip in the strip structured light. Scatter points formed within the stripes such as that in FIG. 31A can be ignored. Referring to Fig. 31A, it shows the first structured light repeating unit and the second structured light repeating unit, assuming that the height difference between them is H ₁ , if the height difference H ₁ is smaller than the first height threshold, the processing object can be determined The pitch angle of the processing object is determined to be qualified, otherwise it is determined that the pitch angle of the processing object is unqualified. If the first structured light repeating unit and the second structured light repeating unit extend along a direction parallel to the axis of any processing object (for example, a single finger), they can be used to determine the roll angle of the processing object. For example, the height difference between the first structured light repeating unit and the second structured light repeating unit corresponding to the processing object may be determined, and the height difference is used to represent the roll angle of the processing object, as shown in FIG. 31B . Fig. 31B shows a schematic diagram of arrangement of structured light repeating units in structured light according to another embodiment of the present application. It can be understood that FIG. 31B only shows the first structured light repeating unit and the second structured light repeating unit arranged in a direction parallel to the axis of the object to be treated, and the fingers are not shown. But the finger corresponding to Figure 31B can be understood as the swing of the finger shown in Figure 31A The placement positions are consistent, so that the structured light repeating units shown in FIG. 31B can be arranged in a direction parallel to the axis of the finger. For example, the height difference between the first structured light repeating unit and the second structured light repeating unit can be calculated, if the height difference is less than the second height threshold, it can be determined that the roll angle of the processing object is qualified, otherwise it can be determined that the processing object roll angle failed. The second height threshold can be set to any suitable value as required. When the structured light repeating unit is a strip, the height of the strip can be expressed by the height of a certain point on the strip or the average height of several points, for example, by the height of the midpoint of the strip.

In the preview handprint image, if the third structured light repeating unit extends along a direction perpendicular to the axis of any processing object (such as a single finger), the height of the two stripe points of the third structured light repeating unit can be determined. Handles the roll angle of the object. If the third structured light repeating unit extends along a direction parallel to the axis of any processing object (such as a single finger), the pitch angle of the processing object can be determined according to the heights of the two stripe points of the third structured light repeating unit. Exemplarily, when the third structured light repeating unit is a stripe, two farther apart stripe points can be selected on the third structured light repeating unit. For the case where the third structured light repeating unit extends along a direction perpendicular to the axis of any treatment object, the pitch angle can be determined in a similar manner, that is, the height difference between these two stripe points can be calculated, expressed by the height difference Handles the roll angle of the object. Fig. 32A shows a schematic diagram of a structured light image according to another embodiment of the present application. As shown in Fig. 32A, by calculating the distance between the first strip point a and the second strip point b on the third structured light repeating unit height difference, if the height difference is less than the third height preset threshold, it can be determined that the roll angle of the finger is qualified; otherwise, it is determined that the roll angle of the processing object is unqualified. The third height threshold can be set to any suitable size as required, for example, 4mm. Similarly, for the case where the third structured light repeating unit extends along a direction parallel to the axis of any processing object, the height difference between the two stripe points on the third structured light repeating unit can be calculated, using the height The difference represents the pitch angle of the processing object, as shown in Fig. 32B. Fig. 32B shows a schematic diagram of arrangement of structured light repeating units in structured light according to another embodiment of the present application. It can be understood that FIG. 32B only shows the third structured light repeating unit arranged in a direction parallel to the axis of the treatment object, and the fingers are not shown. However, the finger corresponding to FIG. 32B can be understood as being placed in the same position as the finger shown in FIG. 32A , so that the structured light repeating units shown in FIG. 32B can be arranged in a direction parallel to the axis of the finger. Referring to FIG. 32B, if the height difference between the two stripe points a' and b' on any third structured light repeating unit is smaller than the fourth height preset threshold, it can be determined that the pitch angle of the finger is qualified, otherwise it can be determined The pitch angle of the processing object is unqualified. The fourth height threshold can be set to any suitable size as required, for example, 4mm.

According to the position and/or shape of the processing object in the unstructured light channel (which may be referred to as an unstructured light channel image) of the previewed handprint image, the position and/or yaw angle of the processing object can be determined. For example, according to the obtained unstructured light channel image of the previewed handprint image, the unstructured light channel image may be segmented to determine a mask or envelope of each processing object. The image segmentation of the unstructured light channel image can be implemented by using any suitable existing or future image segmentation network model. For example, the image segmentation network model may include one or more of the following: fully convolutional network (Fully Convolutional Networks, FCN), U-shaped network (Unet), depth experiment Room (DeepLab) series, V-shaped network (Vnet) and other network models. Exemplarily, the target object in the current unstructured light channel image includes two processing objects, such as the user's double thumbs. Based on the image segmentation, the mask or envelope of the two thumbs in the unstructured light channel image can be obtained. Through the masks or envelopes of the two thumbs, the positions and/or shapes of the two thumbs in the unstructured light channel image can be determined, and then whether the position and/or yaw angle of the processing object is qualified can be judged.

Exemplarily, the position of the mask and/envelope of any processing object (such as the left thumb) can be used to represent the position of the processing object, and the mask and/envelope shape of any processing object (such as the left thumb) can be used Indicates the shape of the processing object. Fig. 33 shows a schematic diagram of processing objects in an unstructured light channel image according to an embodiment of the present application. In an example, it may be determined whether the position of any processing object (that is, the position of its mask and/or envelope) is in the central area of the current unstructured light channel image. The size of the central area can be set as required, and it at least includes the central point of the unstructured light channel image. If any object to be processed (for example, the left thumb) is in the central area of the unstructured light channel image, its position can be determined to be qualified; otherwise, its position can be determined to be unqualified. As shown in FIG. 33 , point r is the center point of the unstructured light channel image. According to the position of the envelope of the processing object, it can be judged that the position of the processing object is not in the central area of the current unstructured light channel image, so its position is unqualified. In another example, different position determination criteria may be set for different fingers. For example, in the case where the target object is four fingers, it can be determined whether the top of the middle finger (which may be the top of the middle finger's mask or envelope) is higher than the first height position on the unstructured light channel image, and if so, determine The position of the middle finger is unqualified; it can also be judged whether the top of the little finger (which can be the top of the mask or envelope of the little finger) is lower than the second height position on the unstructured light channel image, and if so, determine that the position of the little finger is unqualified ; It is also possible to determine whether the leftmost point (which may be the leftmost point of the mask or envelope of the leftmost finger) located on the leftmost finger among the four fingers is located on the left side of the first width position of the unstructured light channel image , if yes, determine that the position of the leftmost finger is unqualified; it can also be judged whether the leftmost point on the rightmost finger among the four fingers (which can be the leftmost point of the mask or envelope of the leftmost finger) It is located on the right side of the second width position of the unstructured light channel image, if yes, it is determined that the position of the rightmost finger is unqualified. The first height position, the second height position, the first width position and the second width position can all be set as required. In an example, the first height position may be 0.1H from the upper edge of the unstructured light channel image, the second height position may be 0.3H from the lower edge of the unstructured light channel image, and the first width position may be a distance from At 0.1W from the left edge of the unstructured light channel image, the second width position can be 0.1W from the right edge of the unstructured light channel image, H is the entire height of the unstructured light channel, and W is the entire width of the unstructured light channel .

In an example, it may be determined whether the shape of any object to be processed (that is, the shape of its mask and/or envelope) in the current unstructured light channel image is tilted relative to the axis of the unstructured light channel image. The axis of the unstructured light channel image may be a center line of the unstructured light channel image extending along a preset direction (for example, the height direction of the image). For example, the inclination angle between the shape of any processing object and the axis of the current unstructured light channel image can be determined, if the inclination angle is less than the preset angle threshold, it can be determined that the yaw angle of the processing object is qualified, otherwise it can be determined the place The yaw angle of the processing object is unqualified. The preset angle threshold can be set to any suitable value as required. Fig. 34 shows a schematic diagram of an unstructured light channel image according to an embodiment of the present application. As shown in Fig. 34, the dotted line M1N1 indicates the axis of the unstructured light channel image, and the dotted line M2N2 indicates the axis of the processing object. The inclination angle between the axis of the processing object and the axis of the current unstructured light channel image is angle α. If the inclination angle α is smaller than the preset angle threshold, it may be determined that the yaw angle of the processing object is qualified, otherwise it is determined that the yaw angle of the processing object is unqualified.

According to the above technical solution, the pitch angle and/or roll angle of the processing object is determined through the structured light repeating unit under the structured light channel; the position of the object to be processed is determined through the position and/or shape of the processing object under the unstructured light channel And/or yaw angle, this method can adopt different judgment methods according to different attitude judgment requirements, so as to ensure the accuracy of judgment results.

Exemplarily, the gesture judging object is a target object including multiple processing objects, and determining whether the posture of the gesture judging object is qualified based on the previewed handprint image may include: The height of the four structured light repeating units determines the roll angle of the target object; or, according to the position and/or shape of the target object in the unstructured light channel of the preview handprint image, determines the position and/or yaw angle of the target object; preview The handprint image is collected under the illumination of an unstructured light source, and the preview of the handprint image includes an unstructured light channel.

In one embodiment, the gesture judgment object may be a target object including a plurality of processing objects. At this time, other attitude indicators in the attitude indicators except the first attitude indicator may be used for the overall calculation. For example, the position and roll angle of the target object are calculated as a whole, instead of the attitude index of each processing object being calculated separately. In this way, calculation speed can be increased. For example, the posture judgment object may include four fingers, that is, index finger, middle finger, ring finger, and little finger. According to the heights of the fourth structured light repeating units corresponding to any two fingers (such as the index finger and the little finger) in the structured light channel of the previewed handprint image, the overall roll angle of the four fingers can be determined. It should be noted that the fourth structured light repeating unit corresponding to the two processing objects is the same structured light repeating unit (for example, the same strip) in the structured light. The fourth structured light repeating unit may extend in a direction perpendicular to the axis of the treatment object. For the method of determining the height of the structured light repeating unit in the case of strips, reference can be made to the above description. According to the height difference between the heights of the fourth structured light repeating units corresponding to the two processing objects, it can be judged whether the overall roll angle of the target object is qualified. For example, if the height difference between the heights of the fourth structured light repeating units corresponding to two processing objects is greater than the fifth height threshold, it can be determined that the roll angle of the target object is unqualified; otherwise, it is qualified. The fifth height threshold can be set to any suitable size as required, for example, 5mm.

Those skilled in the art can understand from the relevant description of "determining the position and/or yaw angle of the processing object according to the position and/or shape of the processing object in the unstructured light channel of the previewed handprint image" in the foregoing embodiments. The implementation of "determining the position and/or yaw angle of the target object according to the position and/or shape of the target object in the unstructured light channel of the preview handprint image" is similar, and for the sake of brevity, it will not be repeated here repeat.

According to the above technical scheme, the structured light channel and unstructured light can be The image under the channel to determine the overall roll angle, position and/or shape of the object of interest. This method can be used to judge whether the posture of the target object is qualified as a whole, and the efficiency is high and the accuracy can be guaranteed.

Exemplarily, the preview handprint image is collected when the structured light source is irradiated, and the roll angle and/or pitch angle of the attitude judgment object is selected according to the number of structured light units included in the structured light channel of the preview handprint image. The fifth structured light repeating unit is determined, the three-dimensional information of the processing object is determined according to a plurality of sixth structured light repeating units selected from the structured light channels included in the structured light channel of the handprint image to be processed, and the fifth structured light The density of the repeat unit is less than the density of the sixth structured light repeat unit.

In one embodiment, structured light is used to irradiate the handprint collection area to obtain a preview image of the handprint. According to the preview handprint image obtained under structured light irradiation, the roll angle and/or pitch angle of the attitude judgment object can be determined through the method described in the above embodiment through the plurality of fifth structured light repeating units contained therein. At the same time, at least part of the sixth structured light repeating unit may be selected from the structured light units included in the structured light channel of the handprint image to be processed for 3D reconstruction to determine the 3D information of the processing object. Wherein the density of the sixth structured light repeating unit is greater than the density of the fifth structured light repeating unit. That is to say, the distribution of structured light repeating units for pose determination is sparser, while the distribution of structured light repeating units for 3D reconstruction is denser. For example, in the case where the structured light repeating unit is a strip, all the strips may be used for 3D reconstruction, and a part of the strips may be sampled to determine the roll angle and/or pitch angle of the attitude judgment object.

According to the above technical solution, the roll angle and/or pitch angle of the attitude judgment object is determined by the fifth structured light repeating unit with low density, and this solution has high calculation efficiency. In addition, the 3D information of the processing object is determined through the sixth structured light repeating unit with high density, which can ensure the accuracy of the determined 3D information.

Exemplarily, determining whether the posture of the gesture judging object is qualified based on the previewed handprint image may include: judging whether the position and/or yaw angle of the posture judging object are qualified; if the position and/or yaw angle of the posture judging object are qualified, then Judging whether the roll angle and/or pitch angle of the attitude judgment object is qualified; if the roll angle and/or pitch angle of the attitude judgment object are qualified, the attitude of the attitude judgment object is qualified.

In one embodiment, the judgment order of each index can be determined based on the computing power consumption calculated by each index, and the index with less computing power consumption is judged first. For example, to determine whether the posture of the posture judging object is qualified based on the previewed handprint image, it may first be judged whether the position and/or yaw angle of the posture judging object is qualified. If the position and/or the yaw angle of the attitude judgment object are qualified, then it is judged whether the roll angle and/or the pitch angle of the attitude judgment object are qualified. If the position and/or yaw angle of the attitude judging object are unqualified, it can be determined that the attitude of the attitude judging object is unqualified. At this time, the judgment can be optionally stopped, and the preview image collection device can optionally be used to re-acquire the preview handprint image. If the roll angle and/or the pitch angle of the attitude judgment object are qualified, it may be determined that the attitude of the attitude judgment object is qualified. In the case that the roll angle and/or pitch angle of the attitude judging object are unqualified, it can be determined that the attitude of the attitude judging object is unqualified, and at this time, the preview image acquisition device can optionally be used Reacquire the preview image of the handprint.

According to the above technical solution, since the algorithm for judging the position and/or yaw angle of the attitude judging object is generally less expensive than the algorithm for judging the roll angle and/or pitch angle, it can be judged sequentially according to the sequence of this scheme, which can reduce Waste of resources, improve efficiency.

Exemplarily, the posture judgment object is the processing object. Before the processing step S3040, the method may further include: judging whether the number of handprint images to be processed corresponding to each processing object included in the target object reaches a preset number threshold, and if so, Then go to the processing step, otherwise, return to at least obtain step S3030; determine the three-dimensional information of the processing object according to the structured light channel of the to-be-processed handprint image corresponding to the processing object, which may include: according to the selected The structured light channel of the handprint image to be processed determines the three-dimensional information of the processing object, and the selected handprint image to be processed is the handprint image to be processed that meets the quality requirements among the preset number threshold handprint images to be processed corresponding to the processing object .

In one embodiment, the target object may include at least one processing object, and for each processing object, it is necessary to acquire enough images to be processed (for example, a preset threshold number), so as to select the processing object therefrom in step S3040 The corresponding image to be processed with the best quality is subjected to subsequent processing. Before processing step S3040, a preset number threshold of handprint images to be processed may be set in advance. The preset number threshold may be any integer greater than 0, such as 5, 6, 8 and so on. Exemplarily, the preset number threshold may be equal to five. For example, in the case that the target object includes two thumbs, for each object to be processed (each thumb), it may be determined whether the number of handprint images corresponding to the object to be processed reaches the preset number threshold 5. Wherein, the to-be-processed handprint image corresponding to each processing object is an image collected when the processing object is in a qualified posture. For the left thumb, if there are currently less than 5 corresponding handprint images to be processed, you can continue to execute the acquisition step S3030 described in the previous embodiment or execute the acquisition step S3010 to the acquisition step S3030 described in the previous embodiment until the left thumb corresponds to the handprint image to be processed. The number of processed handprint images reaches 5. Exemplarily, if a gesture judgment determines that the posture of the processing object is qualified, and within a short period of time after the judgment is qualified, a plurality of target handprint images corresponding to the processing object have been collected and the handprint to be processed is determined. image, then only the acquiring step S3030 may be performed. Exemplarily, the posture can also be re-judged, that is, the entire process of collecting step S3010-posture judging step S3020-obtaining step S3030 can be re-executed to obtain a new handprint image to be processed. Similarly, for the right thumb, if there are currently less than 5 corresponding handprint images to be processed, you can continue to execute the acquisition step S3030 described in the previous embodiment or execute the acquisition step S3010 to the acquisition step S3030 described in the previous embodiment until the right hand The number of to-be-processed handprint images corresponding to the thumb reaches 5. If the number of handprint images to be processed corresponding to the left thumb and the right thumb has reached 5, the processing step S3040 can be started.

From the preset number threshold of handprint images to be processed corresponding to any processing object, select the handprint image to be processed that meets the quality requirements as the selected handprint image to be processed, and according to the structure of the selected handprint image to be processed Optical channel, to determine the 3D information of the object to be processed. Quality requirements can be based on Set as required, for example, it can be the highest quality.

According to the above technical solution, for a single object to be processed, it is ensured that the number of handprint images to be processed reaches the preset number threshold. The to-be-processed images corresponding to the processing object reach the preset number threshold as soon as possible, so the efficiency is relatively high. In addition, according to the above solution, the handprint images to be processed that meet the quality requirements are selected from the preset threshold number of handprint images to be processed corresponding to the processing object, so as to determine the three-dimensional information of the processing object therefrom. The method can ensure that the image quality of the handprint image to be processed for determining the three-dimensional information meets the quality requirements, so as to improve the accuracy of the determined three-dimensional information.

Exemplarily, the posture judgment object is a target object, and before the processing step, the method may further include: judging whether the number of target handprint images corresponding to the target object reaches a preset number threshold, if so, go to the processing step, otherwise , return to perform at least the acquisition step; determining the three-dimensional information of the processing object according to the structured light channel of the handprint image to be processed corresponding to the processing object may include: according to the structured light of the selected handprint image to be processed corresponding to the processing object The channel determines the three-dimensional information of the object to be processed, and the selected handprint image to be processed is the handprint image to be processed that meets the quality requirements among the preset number of threshold handprint images to be processed corresponding to the object to be processed.

In this embodiment, the posture judgment object is the target object. At this time, the target handprint images of each processing object included in the target object must reach the preset number threshold at the same time, because the postures of each processing object are always qualified at the same time. It can be understood that, even if the poses of the processing objects are always qualified at the same time, the target image acquisition devices corresponding to the processing objects may be different. Assuming that the preset number threshold is 3, and the current index finger, middle finger, ring finger and little finger have obtained 3 target fingerprint images (collected by C1, C1, C2, and C2 cameras respectively), it is necessary to perform each The processing object continues to collect 2 target fingerprint images. When continuing to collect, you can continue to execute the acquisition step S3030 described in the previous embodiment or execute the acquisition step S3010 to the acquisition step S3030 described in the previous embodiment until the number of target handprint images corresponding to the target object reaches 3. Exemplarily, if a posture judgment determines that the posture of the target object is qualified, and multiple target handprint images corresponding to the target object have been collected within a short period of time after the judgment is qualified, only the acquisition step S3030 can be performed . Exemplarily, the posture can also be re-judged, that is, the entire process of collecting step S3010-posture judging step S3020-obtaining step S3030 can be re-executed to obtain a new target handprint image.

The implementation of "determining the three-dimensional information of the processing object according to the structured light channel of the handprint image to be processed corresponding to the processing object" is similar to that of the previous embodiment, and will not be repeated here.

According to the above-mentioned technical scheme, it is guaranteed that the number of target handprint images reaches the preset number threshold for the whole target object. If the posture of one processing object is unqualified and the postures of other processing objects are qualified, the overall posture of the target object is unqualified, and no target fingerprint image of any processing object is obtained at this time) and the acquired images are required to reach the preset number threshold, so the target The posture requirements of the object are relatively high, and the quality of the obtained target handprint image is relatively high, which is more helpful for subsequent operations such as handprint recognition. In addition, according to the above solution, the handprint images to be processed that meet the quality requirements are selected from the preset threshold number of handprint images to be processed corresponding to the processing object, so as to determine the three-dimensional information of the processing object therefrom. The method can ensure that the image quality of the handprint image to be processed for determining the three-dimensional information meets the quality requirements, so as to improve the accuracy of the determined three-dimensional information.

According to another aspect of the present application, a non-contact handprint collection device is also provided. FIG. 35 shows a schematic block diagram of a non-contact handprint collection device 3500 according to an embodiment of the present application. As shown in FIG. Computer program instructions, when the computer program instructions are run by the processor 3510, are used to execute the above-mentioned image stitching method 400, the non-contact target object handprint collection method 3000 or the handprint collection method realized by the handprint collection system 200 (that is, the above-mentioned A scheme for acquiring images when the blue light source group emits light with a blue emission scheme, and the green light source group emits light with a green emission scheme).

Those of ordinary skill in the art can understand the implementation of the non-contact handprint collection device and its beneficial effects through the relevant description of the non-contact target object handprint collection method 3000 in the foregoing embodiment. Let me repeat.

In addition, according to the embodiment of the present application, a storage medium is also provided, on which program instructions are stored, and when the program instructions are executed by a computer or a processor, they are used to perform the above image stitching method 400, non-contact target object hand The collection method 3000 of fingerprints or the corresponding steps of the handprint collection method realized by the handprint collection system 200 (that is, the above-mentioned scheme of collecting images when the blue light source group emits light with a blue light-emitting scheme, and the green light source group emits light with a green light-emitting scheme) . The storage medium may include, for example, a memory card of a smart phone, a storage unit of a tablet computer, a hard disk of a personal computer, a read only memory (ROM), an erasable programmable read only memory (EPROM), a portable compact disk read only memory (CD), etc. -ROM), USB memory, or any combination of the above storage media.

In addition, according to an embodiment of the present application, a computer program product is also provided, the computer program product includes a computer program, and the computer program is used to execute the above image stitching method 400, the non-contact target object handprint collection method 3000 or The handprint collection method implemented by the handprint collection system 200 (that is, the above-mentioned scheme for collecting images when the blue light source group emits light in a blue light-emitting scheme, and the green light source group emits light in a green light-emitting scheme).

In addition, according to the embodiment of the present application, a computer program is also provided, and the computer program is used to execute the above-mentioned image stitching method 400, the non-contact target object handprint collection method 3000, or use the handprint collection system 200 to implement The handprint collection method (that is, the above scheme of collecting images when the blue light source group emits light with a blue light-emitting scheme, and the green light source group emits light with a green light-emitting scheme).

In the description provided herein, numerous specific details are set forth. However, it is understood that the embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

Similarly, it should be understood that in order to simplify the application and to facilitate understanding of the various aspects of the application In the description of example embodiments of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof. This method of application, however, is not to be interpreted as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as the corresponding claims reflect, the inventive point lies in that the corresponding technical problem may be solved by using less than all features of a single disclosed embodiment. Thus, the claims following this Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this application.

It will be appreciated by those skilled in the art that all features disclosed in this specification (including accompanying claims, abstract and drawings) and all features of any method or apparatus so disclosed may be used in any combination, except where the features are mutually exclusive. process or unit. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

In addition, those skilled in the art will understand that although some embodiments described herein include some features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the present application. and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

The above is only the specific implementation of the application or the description of the specific implementation. The scope of protection of the application is not limited thereto. Any person familiar with the technical field can easily Any changes or substitutions that come to mind should be covered within the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

A handprint collection system, comprising:

A lighting system, the lighting system includes one or more blue light sources and one or more green light sources, each light source is used to emit light toward the handprint collection area, and the handprint collection area is used to place the object to be photographed, and the photographing Objects include at least one of single finger, double thumb, four fingers, flat palm, and side palm;

A structured light projector, the structured light projector is used to emit structured light to the handprint collection area;

One or more image acquisition devices, the one or more image acquisition devices are used to acquire images of the handprint collection area while the illumination system and the structured light projector emit light to the handprint collection area to obtain a handprint image of the subject;

A processing device, the processing device is used to control the lighting system and the structured light projector to emit light and control the one or more image acquisition devices to collect the handprint image; the processing device is also used to process the Process the handprint images collected by one or more image acquisition devices to obtain the structured light images and illumination light images corresponding to each of the one or more image acquisition devices; perform fusion processing on the illumination light images and based on the structure Transformation processing of the light image to obtain the processed handprint image;

Wherein, the illumination light image includes a blue light image and a green light image, the images used for the fusion process include at least one blue light image and at least one green light image, and the images used for the fusion process correspond to The image acquisition devices are the same, the shooting objects are the same, the shooting time interval is within the interval range, and the shooting lighting conditions are different, and the different lighting conditions include at least one of the color of the light source and the lighting angle of the lighting system.
The system of claim 1, wherein the blue light emitted by the blue light source has a wavelength less than 430 nm, and the green light emitted by the green light source has a wavelength greater than 540 nm.
The system according to claim 1 or 2, wherein the processing device performs fusion processing on the illumination light image and transformation processing based on the structured light image in the following manner:

Perform fusion processing on the blue light image and the green light image in at least one of the illumination light images captured by the same image acquisition device to obtain a fusion image; based on the structured light image corresponding to at least one of the illumination light images, the fusion image Carry out transformation processing, obtain the processed handprint image;

Or, based on the structured light image corresponding to at least one of the illumination light images captured by the same image acquisition device, the blue image in at least one of the illumination light images is transformed to obtain a transformed blue image, based on the same The structured light image corresponding to at least one of the illumination light images captured by the image acquisition device is transformed into a green image in at least one of the illumination light images to obtain a transformed green image, and the transformed blue The image and the converted green image are fused to obtain a processed handprint image.
The system of claim 3, wherein said at least one said illumination light image comprises a first illumination light image taken at a first time instant and a second illumination light image taken at a second time instant; said The processing device performs fusion processing on the blue light image and the green light image in at least one of the illumination light images captured by the same image acquisition device in the following manner to obtain a fusion image:

Fusion processing is performed on the first blue light image and the first green light image in the first illumination light image captured by the same image acquisition device at the first moment to obtain the first blue-green fusion image, which is captured by the same image acquisition device at the second moment Fusion processing is performed on the second blue-light image and the second green-light image in the second illumination light image to obtain a second blue-green fusion image, and fusion is performed on the first blue-green fusion image and the second blue-green fusion image processing to obtain the fused image.
The system of claim 3, wherein said at least one said illumination light image comprises a first illumination light image taken at a first moment and a second illumination image taken at a second moment; said transformed blue image comprising a first transformed blue image determined from a first blue image in said first illuminated light image and a second transformed blue image determined from a second blue image in said second illuminated light image, the The transformed green image includes a first transformed green image determined by a first green image in the first illuminated light image and a second transformed green image determined by a second green image in the second illuminated light image ,

The processing device performs fusion processing on the transformed blue image and the transformed green image in the following manner to obtain a processed handprint image:

performing fusion processing on the first transformed blue image and the first transformed green image to obtain a first transformed fusion image; performing fusion processing on the second transformed blue image and the second transformed green image to obtain The second transformation fusion image is to perform fusion processing on the first transformation fusion image and the second transformation fusion image to obtain the processed handprint image.
The system according to claim 1 or 2, wherein the illumination light image comprises a first illumination light image taken by the same image acquisition device at the first moment and a second illumination image taken at the second moment; Perform fusion processing on the illumination light image and transformation processing based on the structured light image in the following manner to obtain the processed handprint image:

performing fusion processing on the first blue image in the first illumination light image and the first green image in the first illumination light image to obtain a first blue-green fusion image; The second blue image and the second green image in the second illumination light image are fused to obtain a second blue-green fusion image; based on the first structured light image corresponding to the first illumination light image, the Transformation processing is performed on the first blue-green fusion image to obtain a first fusion transformation image; based on the second structured light image corresponding to the second illumination light image, transformation processing is performed on the second blue-green fusion image to obtain a second Fusion and transformation of images: performing fusion processing on the first fusion transformation image and the second fusion transformation image to obtain the processed handprint image.
The system according to any one of claims 1-6, wherein said fusion processing comprises computational fusion and/or neural network model fusion;

The calculation fusion includes: calculating the sharpness index of the pixel at the first position in the image for fusion processing, and using the pixel value of the pixel at the first position with a better sharpness index in the image for fusion processing as the fusion The pixel value of the pixel at the first position in the resulting image;

The neural network model fusion includes: the second position in the image used for fusion processing Input the pixel of the neural network model to obtain the pixel value of the pixel at the second position in the fused image.
The system according to claim 7, wherein said fusion processing comprises computational fusion and neural network model fusion,

The calculation fusion includes: calculating the sharpness index of the pixel at the first position in the image used for fusion processing, and if the gap between the sharpness indexes in the multiple images used for fusion processing is greater than the gap threshold, the The pixel value of the pixel at the first position with a better definition index in the fusion processed image is used as the pixel value of the pixel at the first position in the obtained image after fusion;

The fusion of the neural network model includes: inputting the pixel at the second position in the image for fusion processing into the neural network model to obtain the pixel value of the pixel at the second position in the obtained image after fusion; the second position is used for The position where the difference of the sharpness index among the multiple images subjected to fusion processing is not greater than the difference threshold.
The system according to any one of claims 1-8, wherein the one or more image acquisition devices comprise a plurality of common image acquisition devices jointly photographing the same subject, and the processing device controls the illumination by The light image is fused and processed based on the transformation process of the structured light image to obtain the processed handprint image:

performing fusion processing on the illumination light image, transformation processing based on the structured light image, and splicing the transformed images corresponding to the common image acquisition device to obtain a processed handprint image; the photographed object Including single finger, flat palm or side palm.
The system according to claim 9, wherein the processing device performs fusion processing on the illumination light image, transformation processing based on the structured light image, and transformation processing corresponding to the common image acquisition device in the following ways After the images are spliced, the processed handprint image is obtained:

For each image acquisition device in the plurality of common image acquisition devices, fusion processing is performed on the blue image and the green image in at least one illumination light image corresponding to the image acquisition device to obtain a fusion image corresponding to the image acquisition device ; Based on the structured light image corresponding to the image acquisition device, the fusion image corresponding to the image acquisition device is transformed to obtain the transformed fusion image corresponding to the image acquisition device; the transformation corresponding to the plurality of common image acquisition devices Merge the images for splicing to obtain the processed handprint image;

Or, for each image acquisition device in the plurality of common image acquisition devices, based on the structured light image corresponding to the image acquisition device, the blue image in at least one illumination light image corresponding to the image acquisition device is transformed processing to obtain the converted blue image corresponding to the image acquisition device, and based on the structured light image corresponding to the image acquisition device, perform conversion processing on the green image in at least one illumination light image corresponding to the image acquisition device to obtain the image acquisition transforming the green image corresponding to the device, performing blue-green fusion processing on the transformed blue image corresponding to the image acquisition device and the transformed green image corresponding to the image acquisition device, to obtain a transformed fusion image corresponding to the image acquisition device; The transformation and fusion images corresponding to the common image acquisition device are spliced to obtain the processed handprint image;

Or, for each image acquisition device in the plurality of common image acquisition devices, based on the structured light image corresponding to the image acquisition device, the blue image in at least one illumination light image corresponding to the image acquisition device is transformed processing to obtain the converted blue image corresponding to the image acquisition device, and based on the structured light image corresponding to the image acquisition device, perform conversion processing on the green image in at least one illumination light image corresponding to the image acquisition device to obtain the image acquisition The converted green image corresponding to the device; the converted blue images corresponding to the plurality of common image acquisition devices are spliced to obtain a spliced transformed blue image; the transformed green images corresponding to a plurality of image acquisition devices are spliced to obtain a spliced transformed green image image; performing fusion processing on the mosaic transformed blue image and the mosaic transformed green image to obtain a processed handprint image.
The system according to claim 9 or 10, wherein the structured light projector comprises a light source, a pattern generating unit and a converging lens, the pattern generating unit is arranged in front of the light source, and the light source is used to convert the The pattern on the pattern generation unit is projected onto the projection plane to form a reconstructed structured light pattern and a spliced structured light pattern on the projection plane, and the light transmission of the convergent lens is arranged between the pattern generation unit and the projection plane On the path: the reconstructed structured light pattern is different from the stitched structured light pattern, and there is no border overlap between the reconstructed structured light unit in the reconstructed structured light pattern and the stitched structured light unit in the stitched structured light pattern; The reconstruction structured light unit is used for performing transformation processing based on the structured light image, and the stitching structured light unit is used for stitching the transformed images corresponding to the plurality of common image acquisition devices.
The system according to claim 11, wherein the reconstructed structured light unit and the spliced structured light unit meet at least one of the following conditions:

The reconstructed structured light unit includes stripes;

The spliced structured light unit includes scattered points;

The distribution density of the spliced structured light units is greater than the distribution density of the reconstructed structured light units.
The system according to any one of claims 9-12, wherein the plurality of common image acquisition devices comprises a first common image acquisition device, a second common image acquisition device and a third common image acquisition device, and the first common image acquisition device The optical axis of the collection device is perpendicular to the plane of the handprint collection area facing the plurality of common image collection devices, and the optical axis of the second common image collection device is at an angle to the optical axis of the first common image collection device. A first preset included angle, the optical axis of the third common image capture device and the optical axis of the first common image capture device form a second preset included angle; the photographed object includes a single finger.
The system according to any one of claims 9-13, wherein the plurality of common image acquisition devices comprises a fourth common image acquisition device, a fifth common image acquisition device and a sixth common image acquisition device; The lenses of the common image acquisition device, the fifth common image acquisition device, and the sixth common image acquisition device are located in a predetermined plane and are respectively aimed at a plurality of first sub-regions in the handprint acquisition area, and the multiple Any two adjacent ones of the first sub-regions overlap or adjoin each other; the subject includes a flat palm or a side palm.
The handprint collection system according to any one of claims 1-14, wherein said one or more image collection devices comprise a plurality of independent image collection devices each shooting the same subject, said multiple independent image collection devices The clear imageable object surface subspaces of the acquisition devices partially overlap, and the processing device is further used to determine the clearly imageable objects from the plurality of independent image acquisition devices according to the structured light image and/or the illumination light image collected by the preview image acquisition device A target image acquisition device whose surface subspace matches the position of the subject; the image used for fusion processing is collected by the target image capture device; the subject includes two thumbs or four fingers; the preview image capture device is All image acquisition devices in the plurality of independent image acquisition devices, or the image acquisition device with the largest field of view among the plurality of independent image acquisition devices, or different from the plurality of independent image acquisition devices and having a focal length ratio of the plurality of Standalone image capture unit Shorter image capture unit.
The handprint collection system according to claim 15, wherein said multiple independent image acquisition devices comprise a plurality of first independent image acquisition devices; the lenses of said plurality of first independent image acquisition devices are arranged around the centerline and face to The handprint collection area, wherein, the plurality of first independent image acquisition devices respectively have subspaces that can be clearly imaged within the range of depth of field from the front and back of the best object, and the plurality of first independent image acquisition devices The clearly imageable object surface subspaces partially overlap, the clearly imageable object surface subspaces of the plurality of first independent image acquisition devices together form a clearly imageable total space, and the handprint collection area includes the clearly imageable object surface In the total space, the positions of the optimal object planes of the plurality of first independent image acquisition devices in the handprint acquisition area are different; the focal lengths of the lenses of the plurality of first independent image acquisition devices are different; and/or, The distances from the front ends of the lenses of the plurality of first independent image acquisition devices to the handprint acquisition area are different;

The processing device determines from the plurality of independent image acquisition devices according to the structured light image and/or illumination light image collected by the preview image acquisition device in the following manner, the target image acquisition that can clearly image the subspace of the object surface and matches the position of the object to be photographed device:

Determine the height of the subject according to the structured light image collected by the preview image acquisition device; determine the target image acquisition device that can clearly image the subspace of the object surface and match the height of the subject according to the height of the subject; or, according to the structured light collected by the preview image acquisition device The image determines the height of each finger included in the photographed object; according to the height of each finger, a target image acquisition device that can clearly image the subspace of the object surface and matches the height of each finger is determined.
The handprint collection system according to claim 15, wherein said plurality of independent image acquisition devices comprises a plurality of eighth image acquisition devices, the lenses of said plurality of eighth image acquisition devices are located in a predetermined plane and are respectively aligned A plurality of first sub-regions in the handprint collection area, any adjacent two of the plurality of first sub-regions overlap or adjoin each other;

The processing device determines from the plurality of independent image acquisition devices according to the structured light image and/or illumination light image collected by the preview image acquisition device in the following manner, the target image acquisition that can clearly image the subspace of the object surface and matches the position of the object to be photographed device:

Determine the horizontal position of the subject according to the structured light image and/or illumination light image collected by the preview image acquisition device; determine the subspace of the clearly imaged object surface and the subject according to the height of the subject The target image acquisition device matching the horizontal position of the image;

Or, determine the horizontal position of each finger contained in the subject according to the structured light image and/or illumination light image collected by the preview image acquisition device; determine the subspace of the clearly imageable object surface according to the horizontal position of each finger to match the horizontal position of each finger The target image acquisition device.