WO2021057582A1 - Image matching, 3d imaging and pose recognition method, device, and system - Google Patents

Abstract

Provided are an image matching, 3D imaging and pose recognition method, a device, and a system. The image matching method comprises: acquiring an initial image; and performing detection on the initial image to obtain a distorted region and/or a non-distorted region, and generating a matching result for the distorted region and/or a matching result for the non-distorted region. In the technical solution of the present invention, detection is performed on an initial image to obtain a distorted region and/or a non-distorted region, and then a corresponding scheme is used to perform matching for the distorted region and/or the non-distorted region, such that a precise image matching result can be obtained even when a target object is offset from respective reference planes.

Description

Image matching, 3D imaging and gesture recognition method, device and system Technical field

The invention relates to the technical field of image matching, in particular to methods, devices and systems for image matching, 3D imaging and gesture recognition.

Background technique

When a single or multiple image sensors will collect the image of the target object after the projected image (the projected image is unique in a certain spatial range, or becomes a gradual law and unique in the spatial range, etc.) After being sent to the control unit, the control unit matches a single image collected by a single image sensor with a reference image, or matches two images collected by the left and right image sensors, and then performs 3D imaging or performs posture recognition of the target object according to the matching result.

However, it should be noted that taking two image sensors as an example, when the entire or part of the surface of the object is deflected, bent, etc. with respect to the direction of the chip parallel to the image sensor (ie the reference plane), the two image sensors are caused to collect Asynchronous deformation occurs in the image of the image, such as: some feature parts in the image are deflected to the left and right, and the features in the image are respectively enlarged and reduced; or taking a single image sensor as an example, because the general reference image is parallel to the target object When the image sensor chip or the image sensor chip is at a preset angle, when the target object placement position in the actual operation is offset from the target object placement position in the reference image (ie the reference plane), it will also cause the acquisition Part or all of the images are deformed, making this part of the deformed image and the reference image unable to complete a good match.

Summary of the invention

In view of this, the present invention provides a method, device and system for image matching, 3D imaging and gesture recognition.

The first aspect of the present invention provides an image matching method. The image matching method includes:

Get the initial image;

Detect the deformed area and/or non-deformed area in the initial image, generate a matching result for the deformed area and/or generate a matching result for the non-deformed area.

Further, the detecting the deformed area and/or the non-deformed area in the initial image includes:

Acquiring the current image block in the initial image;

Based on the image characteristics of the current image block, it is detected that the current image block is a current deformed image block or a current non-deformable image block; wherein the non-deformable area is a set of the current non-deformable image block; the deformed area Is the set of the current deformed image blocks.

Further, the detecting the deformed area and/or the non-deformed area in the initial image includes:

Acquiring the current image block in the initial image;

Match the current image block to determine whether a matching result can be generated;

If not, the current image block is the current deformed image block; if yes, the current image block is the current non-deformed image block; wherein, the non-deformed area is a set of the current non-deformed image blocks; the deformed area Is the set of the current deformed image blocks.

Further, the generating a matching result for the deformed region includes:

Obtaining the deformation area;

Converting the deformed area into a reference image;

Matching is performed on the reference image to obtain a matching result for the deformed area.

Further, the converting the deformed area into a reference image includes:

Acquiring the current deformed image block in the deformed area;

Extracting the most value pixel point in each row of the current deformed image block;

Fitting the most value pixel points in each row to obtain a fitting line;

Calculating the image shape variable of the fitted line relative to the reference line;

Based on the image deformation variable, the current deformed image block is converted into the current reference image block.

Further, the converting the deformed area into a reference image includes:

The deformed area is converted into a reference image based on Fourier transform.

Further, the converting the deformed area into a reference image includes:

Acquiring the current deformed image block in the deformed area;

Fitting the current deformed image block to obtain a fitting function;

Based on the fitting function, the current deformed image block is converted into a current reference image block.

Further, the converting the deformed area into a reference image includes:

Acquiring the current deformed image block of the deformed area;

Based on the current deformed image block, generating a deformation variable of the target object;

Based on the deformation variable, convert the current deformed image block into a current reference image block.

Further, the generating a matching result for the deformed region includes:

Sequentially generating image groups of the deformed area after unit deformation occurs;

Matching is performed on the image group to obtain a matching result.

Further, the generating a matching result for the deformed region includes:

Acquiring a pre-generated group of template images in which the deformation area sequentially undergoes unit deformation;

Matching is performed on the template image group to obtain a matching result.

Further, the initial image is an image collected by an image sensor after an image is projected to the target object; wherein the projected image has a periodic gradual change pattern and is unique within a certain spatial range, or is within a certain spatial range It is unique.

The second aspect of the present invention provides a gesture recognition method, the gesture recognition method includes:

The image matching method according to any one of the first aspect; and

According to the matching result, the posture information of the target object is generated.

A third aspect of the present invention provides a 3D imaging method, the 3D imaging method includes:

The image matching method according to any one of the first aspect; and

According to the matching result, a 3D image of the target object is generated.

A fourth aspect of the present invention provides an image matching device, the image matching device includes:

Image acquisition module to acquire the initial image;

The image matching module is used to detect the deformed area and/or non-deformed area in the initial image, generate a matching result for the deformed area and/or generate a matching result for the non-deformed area.

A fifth aspect of the present invention provides a gesture recognition device, and the gesture recognition device includes:

The image matching device of the fourth aspect; and

The posture generation module is used to generate posture information of the target object according to the matching result.

A sixth aspect of the present invention provides a 3D imaging device, the 3D imaging device includes:

The image matching device of the fourth aspect; and

The image generation module is used to generate a 3D image of the target object according to the matching result.

A seventh aspect of the present invention provides a system, which includes: an image projector, an image sensor, and a control unit;

The image projector is used to project an image to a target object;

The image sensor is used to collect the initial image of the target object after the image is projected;

The control unit is configured to implement the image matching method described in the first aspect; the gesture recognition method described in the second aspect; and/or the steps of the 3D imaging method described in the third aspect.

An eighth aspect of the present invention provides a computer device that includes a memory, a processor, and a computer program that is stored in the memory and can run on the processor. When the processor executes the computer program, Implement the steps of the image matching method described in the first aspect; the gesture recognition method described in the second aspect; and/or the 3D imaging method described in the third aspect.

A ninth aspect of the present invention provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image matching method described in the first aspect is implemented; The gesture recognition method described above; and/or the steps of the 3D imaging method described in the third aspect.

By first detecting the deformed area and/or non-deformed area in the initial image, and then matching the detected deformed area and/or non-deformed area in a corresponding way; making the target object change in various relative reference planes Under the circumstance, accurate image matching results can still be obtained.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the following will briefly introduce the embodiments and the accompanying drawings needed in the description of the prior art. Obviously, the drawings in the following description are only some implementations of the present invention. For example, for those of ordinary skill in the art, without creative work, other drawings can be obtained from these drawings.

Fig. 1A is a first schematic structural diagram of an embodiment of a system provided by the present invention; Fig. 1B is a second schematic structural diagram of an embodiment of a system provided by the present invention; Fig. 1C is a third schematic structural diagram of an embodiment of the system provided by the present invention ；

2 is a schematic flowchart of an embodiment of an image matching method provided by the present invention;

3 is a schematic diagram of the first process of an embodiment of detecting a deformed area and/or a non-deformed area in an initial image provided by the present invention;

FIG. 4 is a schematic diagram of a second process of an embodiment of detecting a deformed area and/or a non-deformed area in an initial image provided by the present invention;

5 is a schematic diagram of the first process of an embodiment of generating a matching result for a deformed area provided by the present invention;

6 is a schematic diagram of a second process of an embodiment of generating a matching result for a deformed area provided by the present invention;

FIG. 7 is a schematic diagram of a third process of an embodiment of generating a matching result for a deformed area provided by the present invention;

FIG. 8 is a schematic diagram of the first process of an embodiment of detecting that a current image block is a current deformed image block or a current non-deformable image block based on image features according to the present invention;

FIG. 9 is a schematic diagram of a second process of an embodiment of detecting whether a current image block is a current deformed image block or a current non-deformable image block based on image characteristics according to the present invention;

10 is a schematic diagram of the first process of an embodiment of converting a deformed area into a reference image provided by the present invention;

11 is a schematic diagram of the second process of an embodiment of converting a deformed area into a reference image provided by the present invention;

FIG. 12 is a schematic diagram of the third process of an embodiment of converting a deformed area into a reference image provided by the present invention; FIG.

FIG. 13 is a schematic flowchart of an embodiment of 3D imaging provided by the present invention;

14 is a schematic flowchart of an embodiment of a gesture recognition method provided by the present invention;

15 is a first structural block diagram of an embodiment of a matching device provided by the present invention;

FIG. 16 is a second structural block diagram of an embodiment of a matching device provided by the present invention;

FIG. 17 is a third structural block diagram of an embodiment of a matching device provided by the present invention;

18 is a fourth structural block diagram of an embodiment of a matching device provided by the present invention;

19 is a fifth structural block diagram of an embodiment of a matching device provided by the present invention;

20 is a sixth structural block diagram of an embodiment of a matching device provided by the present invention;

21 is a structural block diagram of an embodiment of a gesture recognition device provided by the present invention;

22 is a structural block diagram of an embodiment of a 3D imaging device provided by the present invention;

Figure 23 is a structural block diagram of an embodiment of a computer device provided by the present invention;

FIG. 24 is a schematic diagram of an embodiment of image conversion provided by the present invention;

FIG. 25A is a first schematic diagram of an embodiment of an initial image provided by the present invention; FIG. 25B is a second schematic diagram of an embodiment of an initial image provided by the present invention; FIG. 25C is a third schematic diagram of an embodiment of an initial image provided by the present invention Figure 25D is a fourth schematic diagram of an embodiment of an initial image provided by the present invention;

Fig. 26A is a schematic diagram of an embodiment of a current preprocessed image block provided by the present invention; Fig. 26B is a schematic diagram of an embodiment of a current preprocessed image block after conversion provided by the present invention; Schematic diagram of the image block;

FIG. 27 is a schematic diagram of intermediate results produced by the method for detecting deformed regions provided by the present invention.

detailed description

The present invention will be further described in detail below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described here are only used to explain the related invention, but not to limit the invention. In addition, it should be noted that, for ease of description, only the parts related to the relevant invention are shown in the drawings.

It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other if there is no conflict. Hereinafter, the present invention will be described in detail with reference to the drawings and in conjunction with the embodiments.

As shown in FIG. 1A, 1B or 1C, an embodiment of the present invention provides a system, which includes an image sensor 11, an image projector 12, and a control unit 13;

The image projector 12 is used to project an image to a target object.

In one embodiment, the image projector 12 can be used to project a periodic gradual and unique image within a certain spatial range to the target object. For details, please refer to the previously applied patent (Publication No. CN107241592A) As described in.

In one embodiment, the periodic gradual change rule may include, but is not limited to, periodically changing sine wave (or cosine wave, etc.) fringes. Specifically, the sine wave does not necessarily fully meet the sine wave standard, and may also be fringes close to sine. For example: incompletely regular sine fringes, or linearly changing sine fringes (also called triangular waves).

Among them, being unique within a certain spatial range means that an image frame of a certain specification moves in any area of the image, and the image in the corresponding area within the image frame is always unique. For example: sine wave stripes with random patterns.

In another embodiment, the image projector 12 can also be used to project a unique image within a certain spatial range, such as a scattered pattern, to the target object.

It should be noted that, in addition to the two images listed above, the image projector 12 can also project any other images that can meet the requirements of subsequent matching.

Specifically, the image projector 12 may be a projector or a laser projector that can project the above-mentioned image that is currently or will be developed in the future.

The image sensor 11 is used to collect the initial image of the target object after the projected image.

The image sensor 11 sends the collected initial image to the control unit 13, the memory or the server, and so on.

Specifically, the image sensor 11 may be, but is not limited to, a camera, a video camera, a scanner, or other devices with related functions (mobile phones, computers, etc.), and so on.

The image sensor 11 may be a group (as shown in FIG. 1A, 1B) or multiple groups (as shown in FIG. 1C) arranged around the target object; wherein, each group of image sensors may include one image sensor (as shown in FIG. 1A). (Shown), 2 image sensors (as shown in Figures 1B and 1C), or more than 2 image sensors (illustration omitted).

The image sensor 11 can be fixedly arranged relative to the target object, or can be movably arranged relative to the target object, which is not limited in this specific embodiment.

The control unit 13 respectively communicates with the image projector 11 and the image sensor 12 in a wired or wireless manner, and communicates with the image projector 11 and the image sensor 12. Wireless methods may include, but are not limited to: 3G/4G, WIFI, Bluetooth, WiMAX, Zigbee, UWB (ultra wideband), and other wireless connection methods that are currently or will be developed in the future.

The control unit 13 may be a part of an independent computing device; it may also be a part of the image projector 11; and/or a part of the image sensor 12. In this specific embodiment, for convenience of description, the control unit 13 is a separate component Show.

For specific definitions of the control unit 13, please refer to the definitions of image matching, 3D imaging and/or gesture recognition methods below. The control unit can be Programmable Logic Controller (PLC), Field-Programmable Gate Array (FPGA), Computer (Personal Computer, PC), Industrial Control Computer (Industrial Personal Computer, IPC) , Smart phone, tablet or server, etc. The control unit generates program instructions according to a pre-fixed program, combined with manually input information, parameters, or data collected by an external image sensor.

As shown in FIG. 2, in one embodiment, the present invention provides an image matching method. Taking the method applied to the system described in the above embodiment as an example, the image matching method includes the following method steps:

Step S110: acquiring an initial image;

Step S120 detects the deformed area and/or the non-deformed area in the initial image, and generates a matching result for the deformed area and/or generates a matching result for the non-deformed area.

The deformed area and/or non-deformed area in the initial image are determined by detection, that is, the initial image may include both deformed areas and non-deformed areas at the same time, or all deformed areas, or all non-deformed areas.

For the detected deformed area and/or non-deformed area, respective methods are used to generate corresponding matching results. Among them, for the non-deformed area, the image matching method described in the following embodiment can be used to directly match the non-deformed area, and for the deformed area, the deformed area needs to be processed (further detailed description will be given in the following embodiment). Then, perform matching according to the image matching method described in the following embodiment.

By detecting the deformed area and/or non-deformed area in the initial image, and then use the corresponding method to match the detected deformed area and/or non-deformed area; making the target object change in various relative reference planes , And still get accurate image matching results.

To facilitate understanding, the steps of the above method are described in further detail below.

Step S110: acquiring an initial image;

Obtain the initial image of the target object collected and sent by the image sensor in real time after the projected image, or obtain the above initial image from the memory or server, or the initial image collected and sent by the image sensor in real time after some processing (such as: cropping, brightness normalization) The initial image formed after transformation, etc.).

In one embodiment, as shown in FIG. 1A, according to the above embodiment, a group of image sensors can be set, each group of image sensors includes 1 image sensor, the initial image is one, and a series of different distances are stored in advance. Download the reference image of the target object after the projected image collected by the image sensor, and subsequently match the initial image with a series of reference images.

In one embodiment, as shown in FIG. 1B, according to the above description, a group of image sensors may be provided, each group of image sensors includes two image sensors 11, and each group of initial images includes a first initial image and a second initial image. Two images.

In one embodiment, as shown in FIG. 1C, N groups of image sensors are set according to the above description, where N is an integer greater than or equal to 2, and each group of images is composed of two image sensors 11, that is, each group of initial The image includes the first initial image and the second initial image.

When the position of the image sensor relative to the target object is fixed, because a single group of image sensors is limited by the field of view, it may not be able to complete the image acquisition of the entire target object at the same time. Therefore, it is necessary to collect the initial image by multiple sets of image sensors that are calibrated with each other, and then Then splice as needed.

Specifically, various image splicing methods currently available or developed in the future can be used, such as: feature-based splicing and region-based splicing.

In one embodiment, the region-based splicing method can start from the gray value of the image to be spliced, and calculate the gray value of an area of the image to be spliced with the same size in the reference image using other methods such as least squares. The difference is compared to the difference to determine the similarity of the overlapping area of the image to be spliced, so as to obtain the range and position of the overlapping area of the spliced image, and then realize image splicing.

In one embodiment, the feature-based stitching is to derive the features of the image through pixels, and then use the image features as a standard to search and match the corresponding feature regions of the overlapping parts.

Specifically, each group of initial images varies according to the number of image sensors. The number of image sensors can be any number greater than or equal to 1. Since the imaging mode of more than 2 image sensors is also a combination of any two of them, the imaging method is The corresponding imaging methods under two image sensors are repeated. Therefore, this specific embodiment only uses a single group of image sensors, and the group of image sensors includes one image sensor or two image sensors as an example for description.

Step S120 detects the deformed area and/or the non-deformed area in the initial image, and generates a matching result for the non-deformed area and/or generates a matching result for the deformed area, respectively.

Among them, the deformed area refers to some areas in the initial image that are inconsistent with the reference image. The deformed area is due to the target object (1 image sensor as an example) or part of the surface on the target object (2 image sensors as an example) relative to the reference surface (taking 2 image sensors as an example, the reference surface can be parallel to the image sensor) The surface of the chip; taking an image sensor as an example, the reference surface can refer to the deflection, bending, etc. of the target object parallel to the shooting of the reference image, such as the surface set in the center, so that the image sensor can collect This part of the image is relative to the reference image (take 2 image sensors as an example, the reference image refers to the image collected when the surface of the target object is parallel to the chip of the image sensor; taking 1 image sensor as an example, the reference image refers to The reference image) is deformed. Specifically, the inconsistency may be manifested as: the deformation area is deflected, stretched, compressed, and/or bent relative to the reference image. For example: Taking the characteristic part of the sine wave in the reference image as an example, the sine wave image in the deformed area is deflected by 2 left (as shown in Fig. 25A), right deflection (as shown in Fig. 25B), and the sine wave in the deformed area Stretching (as shown in FIG. 25C), compression (as shown in FIG. 25D), or bending (illustration omitted), etc.

Among them, the non-deformed area refers to the area in the initial image that is consistent with the reference image.

Specifically, the image matching method may include but is not limited to the following method steps:

In one embodiment, taking two image sensors as an example, the matching is performed between the two initial images. Specifically, the initial image includes a first initial image and a second initial image. When calculating the correspondence between the pixels of the two images, a fixed-size image frame is usually set with the matched pixel as the center. Match the image blocks in the image frame. An n*n image block in the first initial image will be compared with the N image blocks of the same size in the second initial image along the epipolar direction of the two image sensors (N is one of the parallaxes of the two images Search scope). In one embodiment, the method of comparison is to calculate the absolute value of the brightness difference of the corresponding pixels of the two image blocks, and then sum the absolute value to obtain a matching score. From this, N matching scores can be obtained, and the minimum value of these N matching scores can be obtained, and the pixel point in the second initial image corresponding to this minimum value is the matched pixel point in the first initial image Correspondingly, and so on, so as to obtain the matching result of multiple pixels in the two images corresponding to each other.

In one embodiment, one image sensor is taken as an example. The above embodiment is to match between the first initial image and the second initial image, and in the embodiment of one image sensor, the initial The image is matched with the reference image, and the specific matching method can be referred to the above embodiment, which will not be repeated here.

As shown in FIG. 3, in one embodiment, "detecting deformed regions and/or non-deformed regions in the initial image" in step S120 may include the following method steps:

Step S121: acquiring the current image block in the initial image;

According to the above embodiment, the current image block is the image block for current matching.

Step S123, based on the image characteristics of the current image block, detect that the current image block is the current deformed image block or the current non-deformable image block;

Specifically, the initial image includes a deformed area and/or a non-deformed area.

Among them, the deformed area refers to the current set of deformed image blocks; the non-deformed area refers to the current set of non-deformed image blocks.

According to the above embodiments, since the image projected on the surface of the target object has certain characteristics, the deformed area can be detected by judging whether these characteristics are deformed, and then the deformed area can be converted to the image generated when the corresponding reference surface is generated.

For example, according to the above embodiment, the image includes the characteristic part of the fringe with a sinusoidal gradual change. When the sinusoidal fringe is parallel to the reference plane, the horizontal X is the repeated change of multiple cycles, and the vertical Y is aligned, which is Upright state. When deformation occurs, the period of the stripe in the horizontal X direction may be stretched or compressed, and/or the vertical Y direction may be inclined, and the current image block may be deformed according to the changes of these characteristic parts in the image.

The method for detecting whether the current image block is the current deformed image block or the current non-deformable image block will be described in further detail in the following embodiments.

As shown in FIG. 4, in one embodiment, "detecting deformed regions and/or non-deformed regions in the initial image" in step S120 may include the following method steps:

Step S122 obtains the current image block in the initial image;

Step S124: match the current image block; determine whether a matching result can be generated;

If no in step S126, use the current image block as the current deformed image block;

If yes in step S128, use the current image block as the current non-deformable image block.

Specifically, taking two image sensors as an example, according to the image matching method described in step S120 of the above embodiment, an image frame of a preset size is sequentially moved by a unit amount (e.g., the first initial image) along the initial image (e.g., the first initial image). : One pixel), to intercept multiple image blocks for matching; each time the current image block is matched, because the image of the current image block is deformed, the same target will have an asynchronous deformation (such as: the first initial The deformed area of the image is stretched and deformed, and the corresponding same deformed area of the second initial image is compressed and deformed), resulting in the current image block unable to complete a good match, so the current image block that cannot be matched can be regarded as the current A deformed image block, and the current image block that can be matched is the current non-deformable image block, the set of multiple current deformed image blocks is the deformed area, and the set of multiple current non-deformable image blocks is the non-deformed area.

In the same way, when it is an image sensor, when the actual target object and the target object position of the reference image are tilted, etc., it will also make the actual acquired initial image relative to the reference image (ie reference image). Deformation, making it difficult to complete a good match. It should be noted that, usually in this case, the entire image area corresponding to the target object is a deformed area because the entire target object is tilted relative to the reference plane.

It should be noted that the matching result for the current non-deformable image block has been generated in the process of "matching the current image block" in step S124.

Specifically, the "generating a matching result for the deformed area" in the above step S120 can be implemented through but not limited to the following method steps:

As shown in FIG. 5, in one embodiment, "generating a matching result for a deformed area" in step S120 may include the following method steps:

Step S221: Obtain a deformed area;

Specifically, according to the above embodiment, the deformed area may be a collection of multiple current deformed image blocks, so the current deformed image block can be obtained, and the current deformed image block can be converted into the current reference image block in the following step S222.

Step S222: Convert the deformed area into a reference image;

The method of converting the deformed area into the reference image will be described in further detail in the following embodiments.

Step S223 performs matching on the reference image to obtain a matching result for the deformed area.

Then the deformed area is replaced with a reference image, and the reference image is matched. For the matching method, please refer to the relevant description in the image matching method in the above embodiment, which will not be repeated here.

As shown in FIG. 6, in one embodiment, "generating a matching result for the deformed area" in step S120 may include the following method steps:

Step S321 obtains a template image group in which the pre-generated deformation area has a unit deformation in sequence;

In an embodiment, the template image group may be a pre-generated image template group obtained after the current deformed image block sequentially undergoes at least one unit deformation amount. In one embodiment, in order to perform matching for each current image block, a current template image group is generated in advance corresponding to each current image block.

Specifically, the size of each template corresponds to the image block;

Specifically, for example, take the deflection angle of the target object's surface relative to the reference plane as the deformation variable. For example, the plane inclination angle we need to calculate is on the X axis: -60 degrees to +60 degrees, and the Y axis: -60 degrees. From degrees to +60 degrees, every 20 degrees is a unit, the sampling angle is: -60, -40, -20, 0, 20, 40, 60, a total of 7 cases, the XY tilt case is 7*7=49 Templates. It should be noted that, in addition to 20 degrees as the deflection unit, you can set the deflection unit at any angle according to your needs. The smaller the deflection unit, the higher the matching accuracy, but the more times you need to match, so the cost of time or hardware is required. Will be higher.

In an embodiment, the template image can be generated in advance according to the deflection unit, combined with the spatial position conversion information of the image projector and the image sensor; or according to the method of fitting function in the embodiment shown in FIG. 11, the corresponding Template image.

Step S322 performs matching on the template image group to obtain a matching result.

To facilitate understanding, the following takes the matching of the first initial image and the second initial image as an example for further detailed description. For example, the template image corresponding to the current image block in the first initial image is sequentially obtained (for example: deflection -60, -40, -20, 0, 20, 40, and 60 template images), the corresponding second initial image is located on the same epipolar line, and it can include multiple image blocks to be matched. Each template image and the second initial The sum of the absolute values of the gray-scale differences between the image blocks to be matched in the image corresponds to the second initial image with the smallest sum of the absolute values of the gray-scale differences and the current image block of the first initial image. In order to match the image blocks, the pixels corresponding to the two matching image blocks are matched pixels, and the matching result is obtained.

It should be noted that when only one initial image is included, the initial image and the reference image can be matched according to the method described in the above embodiment. At this time, the reference image can be regarded as the second initial image to generate the matching result. .

In an embodiment, the template image group can be generated in advance according to the deflection unit, combined with the spatial position conversion information of the image projector and the image sensor; or according to the method of fitting function in the embodiment shown in FIG. 11, the corresponding Set of template images.

As shown in FIG. 7, in one embodiment, "generating a matching result for the deformed area" in step S120 may include the following method steps:

Step S421 sequentially generates image groups after unit deformation of the deformed area occurs;

Step S422 performs matching on the image group to obtain a matching result.

In one embodiment, taking two image sensors as an example, the deformed image block group formed after the current image block in the first initial image is deflected by a unit angle and the plurality of waiting images located on the same epipolar line in the second initial image are sequentially acquired. The matching image blocks are matched, and the sum of the absolute values of the gray-scale differences between each deformed image block and the image block to be matched in the second initial image is calculated, and then the second one with the smallest sum of the absolute values of the gray-scale differences is calculated. The to-be-matched image block of the initial image and the current image block of the first initial image are matched image blocks, so the pixels corresponding to the two matched image blocks are matched pixels, thereby obtaining the matching result. For other related descriptions, reference can be made to the template image matching method in the above embodiment, which will not be repeated here.

In an embodiment, the image group after unit deformation can be generated according to the deflection unit, combined with the spatial position conversion information of the image projector and the image sensor; it can also be based on the fitting function in the embodiment shown in FIG. 9 below. Method to generate the corresponding image group.

Specifically, in step S123, based on the image characteristics of the current image block, detecting that the current image block is the current deformed image block or the current non-deformable image block can be implemented by, but not limited to, the following method steps:

As shown in FIG. 8, in an embodiment, step S123 may include the following method steps:

S1231 obtains the current image block;

Specifically, according to the above, the image frame can be moved along the epipolar direction of the image sensor on the initial image, and each time it moves one unit, the current image on the initial image corresponding to the image frame is the current image block.

In one embodiment, according to the following embodiment, when it is necessary to further convert the current deformed image block into a reference image based on the detection method, there may be a blank part on the edge of the current deformed image block after the conversion (as shown in FIG. 26B As shown), in order to ensure the integrity of the content of the current image block after conversion, it is usually necessary to obtain a current image block whose size is larger than the actual matching image block (as shown in Figure 26A), and then perform the conversion. The latter image is cropped to obtain the current deformed image block completely converted into the reference image (as shown in FIG. 26C).

S1233 extracting the most value pixel point (for example: the highest gray value point) in each row of the current image block;

Specifically, according to the above embodiment, taking the image including a sine wave image as an example, due to the gradual change rule, if there is no deformation, the peak or trough of each sine wave should be on the same reference line. The pixel point at the peak or valley is the maximum value pixel point, for example, the gray value is the highest or the lowest.

S1235 fits the most value pixel points in each row to obtain the current fitting line of the current image block;

Fit the most value points in each row to a line (as shown in Figure 27), and the two faces of the target object can get lines L1 and L2 respectively.

S1237 detects the current image shape variable of the current fitted line relative to the reference line;

Detect L1 and L2, where L1 is shifted from the reference line, while L2 is not shifted.

S1239, according to the current image shape variable, detects that the current image block is the current deformed image block or the current non-deformable image block.

In one embodiment, it can be determined whether the deformation amount is greater than or equal to a certain threshold; if so, the current image block is a deformed image block; if not, the current image block is an indeformable image block.

Theoretically, when there is no deformation, the deformation of the fitted line and the baseline is zero. However, due to the existence of various errors, it is often not accurately represented as zero. Therefore, a threshold can be preset. If the deformation is less than When it is equal to or less than the threshold (in one embodiment, the threshold may also be a theoretical value of zero), the current image block is considered to be the current non-deformed image block; if it is greater than or equal to a certain threshold, the current image block is determined Is the current deformed image block, and the shape variable of the fitted line relative to the reference line is the current image shape variable of the current image block.

As shown in FIG. 9, in an embodiment, step S123 may include the following method steps:

S1232 obtains the current image block;

Similarly, according to the above embodiment, in one embodiment, in order to ensure the integrity of the content of the current image block after conversion, it is usually necessary to obtain a current image block whose size is larger than the actual required matching image block. .

S1234 fits the current image block to obtain a fitting function;

In one embodiment, the current image block is fitted to obtain a fitting function. Taking the image including periodic sinusoidal stripes as an example, the fitting function can be: Z=Asin(BX+CY+D)+E ; Suppose the pixel value intensity of a certain pixel in the current initial image block is Z, the abscissa is X, and the ordinate is Y; where A is the amplitude of the sine function; 360/B is the period of the function's brightness change (unit: Pixels), C represents the inclination of the pixel in the Y direction. When there is no deformation, C is 0. In the Y direction, each line of the image above is shifted along the positive X direction in the horizontal X direction relative to each line of the image below. B pixels, the inclination angle is arccotangent (C/B); D is the translation amount of the function in the lateral X direction; E is the translation amount of the function in the Z direction; where A, D and E are fixed values.

S1236, according to the fitting function, detects whether the current image block is a deformed image block or a non-deformable image block.

According to the above embodiment, C represents the inclination of the pixel in the Y direction. When no deformation occurs, C is 0;

In addition, it can be known from the fitting function that arccotangent (C/B) is the image shape variable of the current deformed image block. In one embodiment, it can be determined whether the image deformation amount is greater than or equal to a certain threshold; if so, the current image block is a deformed image block; if not, the current image block is a non-deformable image block.

Specifically, in step S222, converting the deformed area into a reference image can be achieved through but not limited to the following method steps:

As shown in FIG. 10, in an embodiment, step S222 may include the following method steps:

Step S2221 obtains the current deformed image block;

Step S2222 extracts the most value pixel points in each row of the current deformed image block;

Step S2223 fits the most value pixel points in each row to obtain the current fitted line of the current image block;

Step S2224 detects the current image shape variable of the current fitting line relative to the reference line;

It should be noted that the detailed description of the method steps in step S2221 to step S2224 can refer to the description of step S1231 to step S1237 in step S123 in the above embodiment. In addition, in an embodiment, when step S123 includes the above-mentioned steps S1231-step S1237, then steps S2221-step S2224 here can be omitted.

Step S2225 converts the current deformed image block into the current reference image block based on the current image shape variable.

As shown in FIG. 11, in an embodiment, step S222 may include the following method steps:

Step S2231 obtains the current image block;

Step S2232 performs fitting on the current image block to obtain the current fitting function;

It should be noted that the detailed description of the method steps in step S2231 to step S2232 can be found in the description of step S1232 to step S1234 in the above embodiment. In addition, in an embodiment, when step S123 includes the above-mentioned steps S1232-step S1234, then steps S2231-step S2232 here can be omitted.

Step S2233 converts the current deformed image block into the current reference image block based on the current fitting function;

The deformed area can be converted based on the fitting function, that is, the period of the deformed area is the reference period, and the C of the image is 0, so as to obtain the deformed area after conversion.

To facilitate understanding, the above method will be further described in detail with a specific example below.

Assuming that the size of the image block to be acquired is (width W: 33, height H: 33), since there will be blanks at the edges of the converted image (as shown in Figure 26B), in order to ensure the integrity of the image block content, it is usually necessary to intercept An initial image block larger than the size of the image block, for example, as shown in Figure 26A, a wider area is selected as the initial image block (W: 63, H: 33), where the center of the initial image block Form a rectangular frame, the size of the rectangular frame corresponds to the size of the image block (W: 33, H: 33); in addition, the initial image block can also be any other size, as long as it is larger than the preset image block And ensure that the content in the converted image block is complete.

Specifically, first, with the rectangular frame as the center, the initial image block is fitted with a three-dimensional curve according to the above fitting function.

The fitted function is: Z=55.24×sin(20.73×X-4.07×Y+159.58)+97.85.

Assuming that the parameters of the reference image are: B=20, C=0, the period of the reference image is 360/20 (unit: pixels); and according to the above function, B=20.73, the period of the initial image that needs to be converted is 360/ 20.73 (unit: pixels), that is, the initial image is compressed and deformed in the horizontal direction X, so the initial image needs to be stretched in the horizontal direction X, and the stretching coefficient is: 20.73/20=1.0365. In addition, since C=0 of the reference image, it is necessary to perform a tilt transformation on the stretched image. The transformation method is as follows: Since the height of the image is H: 33, take the middle row (that is, the 17th row) without transformation. The middle row 17 is used as the 0 point reference, and the number of other rows is i, and the i-th row is translated along the positive direction of the X axis by -(i-17)*(C/B) pixels. After all rows are traversed, the transformation is completed. such as:

The whole line of the 18th line is translated along the positive direction of the X axis -(18-17)*C/B=-C/B, that is 4.07/20=0.2035 pixels, the whole line of the 19th line is translated along the positive direction of the X axis-2×C/ B=0.47 pixels and so on. After completing the above two changes, the initial image is converted, and the converted initial image becomes as shown in FIG. 26B.

Further, the area within the rectangular frame (W: 33, H: 33) located in the middle of the deformed initial image is intercepted as the converted image block, and the converted image block after the interception is shown in FIG. 26C.

As shown in FIG. 12, in an embodiment, step S222 may include the following method steps:

Step S2241 obtains the current deformed image block;

Step S2242 generates the current deformation of the target object based on the current deformation image block;

Step S2243, based on the current deformation amount, converts the current deformed image block into the current reference image block;

For example, as shown in FIG. 24, take a certain point A on a certain slope L that is spaced relative to the first image sensor 111 and the second image sensor 112 as an example. To simplify the description, only point A and the first image sensor 111 are taken as Examples are explained. According to the pre-calibrated results, the actual spatial position coordinates of point A in the image corresponding to the image sensor coordinate system can be obtained according to the initial image; according to the deformation of point A relative to the reference plane L'(for example: the deformation can be implemented according to the above The image shape variable obtained in the example is converted based on the calibration result of the image sensor; or obtained based on the template image group described in the above embodiment), then the point A corresponding to the direction of the projected image of the image projector 12 can be calculated The position coordinates of the virtual point A'projected on the reference plane O in the image sensor coordinate system. According to the conversion between the first and second image sensor coordinate systems and the first and second image coordinate systems, the positions A'can be obtained respectively The coordinates in the first and second image coordinate systems, and so on, can obtain the corrected image of the deformed area.

In an embodiment, step S222 may be based on Fourier transform to convert the deformed area into a reference image.

As shown in FIG. 13, in one embodiment, a 3D imaging method is further provided, and the 3D imaging method includes the matching method described in the above embodiment, and further includes the steps:

S130 generates a 3D image of the target object according to the matching result.

According to the matching result, for each matching point pair based on the triangulation algorithm or the corresponding reference image, the posture information of the corresponding point of the matching point pair in the three-dimensional space can be obtained, based on the multiple matching point pairs included in the matching result , That is, you can draw a 3D image of the object in a three-dimensional space.

Specifically, the posture information can be 3d coordinates in a preset coordinate system for the target, and the motion of a rigid body in a 3-dimensional space can be described by 3d coordinates (a total of 6 degrees of freedom). Specifically, it can be divided into rotation and translation, each It is 3 degrees of freedom. The translation of a rigid body in a 3-dimensional space is an ordinary linear transformation, and a 3x1 vector can be used to describe the translation position; and the commonly used description methods of rotation pose include but are not limited to: rotation matrix, rotation vector, quaternion, Euler angle and Lie algebra.

As shown in FIG. 14, in one embodiment, a gesture recognition method is also provided. The gesture recognition method includes the matching method described in the above embodiment, and further includes the steps:

S140 generates a posture recognition result of the target object according to the matching result.

According to the matching result, each matching point pair is based on the triangulation algorithm or based on the corresponding reference image, and the posture information of the corresponding point in the three-dimensional space of the matching point pair can be obtained, based on the posture of one or more three-dimensional space points Information to obtain the result of the posture recognition of the target object. Specifically, the result of the posture recognition of the target object may be the posture information representing the position and posture of the entire object or the posture information of a certain target position associated with the target object (for example, the target position may be located on the target object or on the including frame of the target object). .

Or according to the point cloud image obtained in the above embodiment, based on the method of Linemod, the posture information of the target object can be obtained. The Linemod method refers to storing the corresponding point cloud images at multiple angles based on the 3D model of the target object (such as: CAD model) in advance, and matching the point cloud image obtained in the above embodiment with the image in the image library to determine The posture information of the target object.

It should be understood that although the various steps in the flowcharts of FIGS. 1-14 are displayed in sequence as indicated by the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless specifically stated in this article, there is no strict order for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in Figures 1-14 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. These sub-steps or stages The execution order of is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

As shown in FIG. 15, in one embodiment, an image matching device is provided, and the image matching device includes:

The image acquisition module 110 is used to acquire an initial image;

The result generating module 120 is configured to detect the deformed area and/or the non-deformed area in the initial image, generate a matching result for the deformed area and/or generate a matching result for the non-deformed area.

As shown in FIG. 16, in one embodiment, the above result generation module 120 includes:

The image acquisition unit 121 is configured to acquire the current image block in the initial image;

The image detection unit 123 is configured to detect whether the current image block is a current deformed image block or a current non-deformable image block based on the image characteristics of the current image block;

As shown in FIG. 17, in one embodiment, the above result generation module 120 includes:

The image acquisition unit 122 is configured to acquire the current image block in the initial image;

The image matching unit 124 is used to match the current image block, and the result judgment unit 126 is used to judge whether a matching result can be generated;

The image determining unit 128 is configured to, if not, the current image block is the current deformed image block; if so, the current image block is the current non-deformable image block.

As shown in FIG. 18, in one embodiment, the above result generation module 120 includes:

The deformation acquiring unit 221 is configured to acquire the deformation area;

The deformation conversion unit 222 is configured to convert the deformation area into a reference image;

The reference matching unit 223 is configured to perform matching on the reference image to obtain a matching result for the deformed area.

As shown in FIG. 19, in one embodiment, the above result generating module 120 includes:

The template obtaining unit 321 is configured to obtain a template image group in which the pre-generated deformation area has a unit deformation in sequence;

The template matching unit 322 is configured to perform matching on the template image group to obtain a matching result.

As shown in FIG. 20, in one embodiment, the above result generation module 120 includes:

The image generating unit 421 is configured to sequentially generate image groups after unit deformation of the deformed area occurs;

The image matching unit 422 is configured to perform matching on the image group to obtain a matching result.

Further, in an embodiment, the image detection unit 123 includes:

The image acquisition unit 1231 is used to acquire the current image block;

The maximum value extraction unit 1233 is used to extract the maximum value pixel points in each row of the current image block;

The best value fitting unit 1235 is used to fit the best value pixel points in each row to obtain the current fitting line of the current image block;

The deformation detection unit 1237 is used to detect the current image deformation of the current fitting line relative to the reference line;

The image detection unit 1239 is configured to detect whether the current image block is the current deformed image block or the current non-deformable image block according to the current image deformation.

Further, in an embodiment, the image detection unit 123 includes:

The image acquisition unit 1232 acquires the current image block;

The function extraction unit 1234 fits the current image block to obtain a fitting function;

The image detection unit 1236 detects whether the current image block is the current deformed image block or the current non-deformed image block according to the fitting function.

Further, in an embodiment, the deformation conversion unit 222 includes:

The image acquiring unit 2221 is used to acquire the current deformed image block in the deformed area;

The maximum value extraction unit 2222 is configured to extract the maximum value pixel points in each row of the current deformed image block;

The best value fitting unit 2223 is configured to fit the best value pixel points in each row to obtain the current fitted line;

The deformation calculation unit 2224 is used to calculate the current image deformation of the current fitting line relative to the reference line;

The image conversion unit 2225 is configured to convert the current deformed image block into the current reference image block based on the current image shape variable.

Further, in an embodiment, the deformation conversion unit 222 includes:

The image acquiring unit 2231 is used to acquire the current deformed image block in the deformed area;

The function extraction part 2232 is used for fitting the current deformed image block to obtain the current fitting function;

The image conversion unit 2233 is configured to convert the current deformed image block into the current reference image block based on the current fitting function.

Further, in an embodiment, the deformation conversion unit 222 includes:

The image acquiring unit 2241 is used to acquire the current deformed image block of the deformed area;

The deformation generating unit 2242 is configured to generate the current deformation of the target object based on the current deformed image block;

The image conversion unit 2243 is configured to convert the current deformed image block into the current reference image block based on the current deformation amount.

Further, in an embodiment, the deformation conversion unit 222 includes:

The image conversion unit 2251 is configured to convert the deformed area into a reference image based on Fourier transform.

As shown in FIG. 21, in one embodiment, a gesture recognition device is provided, and the gesture recognition device includes:

The above-mentioned image matching device; and

The posture generation module 130 is configured to generate posture information of the target object according to the matching result.

As shown in FIG. 22, in one embodiment, a 3D imaging device is provided, and the 3D imaging device includes:

The above-mentioned image matching module; and

The 3D imaging module 140 is configured to generate a 3D image of the target object according to the matching result.

For the specific definitions of the above-mentioned image matching device, 3D imaging device, and gesture recognition device, please refer to the above definitions of the image matching method, 3D imaging method, and gesture recognition method respectively, which will not be repeated here. Each module in each of the above-mentioned devices may be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

In other embodiments, a computer-readable storage medium is also provided, the computer-readable storage medium stores a computer program, and the computer program implements the image matching method described in the above embodiments when the computer program is executed by a processor, Steps of 3D imaging method and/or gesture recognition method.

For the description of the image matching method, the 3D imaging method, and/or the gesture recognition method, please refer to the above embodiment, which will not be repeated here.

As shown in FIG. 23, in some other embodiments, a computer device is also provided. The computer device includes a memory, a processor, and a computer program that is stored in the memory and can run on the processor. When the processor executes the computer program, the steps of the image matching method, 3D imaging method, and/or gesture recognition method described in the above embodiments are implemented.

For the description of the image matching method, the 3D imaging method, and/or the gesture recognition method, please refer to the above embodiment, which will not be repeated here.

Take computers and industrial control computers as examples. Industrial control computers have important computer attributes and characteristics. Therefore, they all have computer central processing unit (CPU), hard disk, memory and other internal memories, as well as plug-in hard disks. External storage such as Smart Media Card (SMC), Secure Digital (SD) Card, Flash Card (Flash Card), etc., with operating system, control network and protocol, computing power, and friendly man-machine interface, It is to provide reliable, embedded, intelligent computers and industrial control computers for other structures/equipment/systems.

Exemplarily, the computer program may be divided into one or more modules/units, and the one or more modules/units are stored in the memory and executed by the processor to complete the present invention. The one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the control unit.

The so-called processor can be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The memory may be a storage device built into the terminal, such as a hard disk or a memory. The memory may also be an external storage device of the control unit, such as a plug-in hard disk equipped on the control unit, a Smart Media Card (SMC), a Secure Digital (SD) card, and a flash memory. Card (Flash Card), etc. Further, the memory may also include not only an internal storage unit of the control unit, but also an external storage device. The memory is used to store the computer program and other programs and data required by the terminal. The memory can also be used to temporarily store data that has been output or will be output.

Those skilled in the art can understand that FIG. 23 is only an example of a computer device, and does not constitute a limitation on the computer device. It may include more or less components than shown in the figure, or a combination of certain components, or different components, such as The control unit may also include input and output devices, network access devices, buses, and the like.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered as going beyond the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the embodiments of each device described above are only illustrative. For example, the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or Components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated module/unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, it can implement the steps of the foregoing method embodiments. . Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to the legislation and patent practice, the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

When an element is expressed as being "fixed to" another element, it may be directly on the other element, or there may be one or more intermediate elements in between, prefabricated integrally with the other element. When an element is said to be "connected" to another element, it can be directly connected to the other element, or there may be one or more intervening elements in between. The terms "vertical", "horizontal", "left", "right", "inner", "outer" and similar expressions used in this specification are for illustrative purposes only.

Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by those skilled in the technical field of the present invention. What is used in the description of the present invention in this specification is only for the purpose of describing specific embodiments, and is not used to limit the present invention.

The term "and/or" in this article is only an association relationship describing the associated objects, which means that there can be three relationships, for example: A and/or B, which can mean that A exists alone, A and B exist at the same time, and B exists alone. These three situations. In addition, the character "/" in this text generally indicates that the associated objects before and after are in an "or" relationship.

The terms "first", "second", "third", "S110", "S1120", "S130", etc. (if any) in the claims and description of the present invention and the above-mentioned drawings are used to distinguish Similar objects are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in a sequence other than the content illustrated or described herein. In addition, the terms "include", "have" and any of their variations are intended to cover non-exclusive inclusion. For example, a process, method, or system that includes a series of steps or modules is not necessarily limited to those clearly listed steps or modules, but includes those that are not clearly listed or are inherent to these processes, methods, systems, products, or robots. Other steps or modules.

It should be noted that those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the involved structures and modules are not necessarily required by the present invention.

The image matching, 3D imaging, and gesture recognition methods, devices, and systems provided by the embodiments of the present invention are described in detail above, but the descriptions of the above embodiments are only used to help understand the method and core ideas of the present invention, and should not be understood as Restrictions on the invention. Those skilled in the art, based on the idea of the present invention, can easily conceive of changes or substitutions within the technical scope disclosed by the present invention, shall be covered by the protection scope of the present invention.

Claims (19)

1. An image matching method, characterized in that the image matching method includes:

   Get the initial image;

   Detect the deformed area and/or non-deformed area in the initial image, generate a matching result for the deformed area and/or generate a matching result for the non-deformed area.
2. The image matching method according to claim 1, wherein the detecting the deformed area and/or the non-deformed area in the initial image comprises:

   Acquiring the current image block in the initial image;

   Based on the image characteristics of the current image block, it is detected that the current image block is a current deformed image block or a current non-deformable image block; wherein the non-deformable area is a set of the current non-deformable image block; the deformed area Is the set of the current deformed image blocks.
3. The image matching method according to claim 1, wherein the detecting the deformed area and/or the non-deformed area in the initial image comprises:

   Acquiring the current image block in the initial image;

   Match the current image block to determine whether a matching result can be generated;

   If not, the current image block is the current deformed image block; if yes, the current image block is the current non-deformed image block; wherein, the non-deformed area is a set of the current non-deformed image blocks; the deformed area Is the set of the current deformed image blocks.
4. The image matching method according to claim 1 or 2 or 3, wherein the generating a matching result for the deformed area comprises:

   Obtaining the deformation area;

   Converting the deformed area into a reference image;

   Matching is performed on the reference image to obtain a matching result for the deformed area.
5. The image matching method according to claim 4, wherein said converting said deformed area into a reference image comprises:

   Acquiring the current deformed image block in the deformed area;

   Extracting the most value pixel point in each row of the current deformed image block;

   Fitting the most value pixel points in each row to obtain a fitting line;

   Calculating the image shape variable of the fitted line relative to the reference line;

   Based on the image deformation variable, the current deformed image block is converted into the current reference image block.
6. The image matching method according to claim 4, wherein said converting said deformed area into a reference image comprises:

   The deformed area is converted into a reference image based on Fourier transform.
7. The image matching method according to claim 4, wherein said converting said deformed area into a reference image comprises:

   Acquiring the current deformed image block in the deformed area;

   Fitting the current deformed image block to obtain a fitting function;

   Based on the fitting function, the current deformed image block is converted into a current reference image block.
8. The image matching method according to claim 4, wherein said converting said deformed area into a reference image comprises:

   Acquiring the current deformed image block of the deformed area;

   Based on the current deformed image block, generating a deformation variable of the target object;

   Based on the deformation variable, convert the current deformed image block into a current reference image block.
9. The image matching method according to claim 1 or 2 or 3, wherein the generating a matching result for the deformed area comprises:

   Sequentially generating image groups of the deformed area after unit deformation occurs;

   Matching is performed on the image group to obtain a matching result.
10. The image matching method according to claim 1 or 2 or 3, wherein the generating a matching result for the deformed area comprises:

    Acquiring a pre-generated group of template images in which the deformation area sequentially undergoes unit deformation;

    Matching is performed on the template image group to obtain a matching result.
11. The image matching method according to claim 1 or 2 or 3, wherein the initial image is an image collected by an image sensor and projected to a target object; wherein the projected image becomes a periodic gradient Regular and unique within a certain space, or unique within a certain space.
12. A gesture recognition method, characterized in that the gesture recognition method includes:

    The image matching method according to any one of claims 1-11; and

    According to the matching result, the posture information of the target object is generated.
13. A 3D imaging method, characterized in that, the 3D imaging method includes:

    The image matching method according to any one of claims 1-11; and

    According to the matching result, a 3D image of the target object is generated.
14. An image matching device, characterized in that the image matching device includes:

    Image acquisition module to acquire the initial image;

    The image matching module is used to detect the deformed area and/or non-deformed area in the initial image, generate a matching result for the deformed area and/or generate a matching result for the non-deformed area.
15. A gesture recognition device, characterized in that the gesture recognition device includes:

    The image matching device of claim 14; and

    The posture generation module is used to generate posture information of the target object according to the matching result.
16. A 3D imaging device, characterized in that, the 3D imaging device comprises:

    The image matching device of claim 14; and

    The image generation module is used to generate a 3D image of the target object according to the matching result.
17. A system, characterized in that the system includes: an image projector, an image sensor, and a control unit;

    The image projector is used to project an image to a target object;

    The image sensor is used to collect the initial image of the target object after the image is projected;

    The control unit is configured to implement the image matching method according to any one of claims 1-11; the gesture recognition method according to claim 12; and/or the steps of the 3D imaging method according to claim 13.
18. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the rights when the computer program is executed. The image matching method according to any one of claims 1-11; the gesture recognition method according to claim 12; and/or the steps of the 3D imaging method according to claim 13.
19. A computer-readable storage medium storing a computer program, wherein the computer program implements the image matching method according to any one of claims 1-11 when the computer program is executed by a processor; The gesture recognition method according to claim 12; and/or the steps of the 3D imaging method according to claim 13.

Images